Princess Beatrice and Princess Eugenie grew up celebrating the holidays in Sandringham with their parents and the other members of the royal family. But in the Christmas of 1992, everything changed for the royal siblings.

Sarah Ferguson and Prince Andrew announced their separation that year. And even though the Duchess of York was still considered part of the royal family after her split from the Duke of York, things changed following a major scandal.

Fergie was photographed with business tycoon John Bryan, and a photo of the latter sucking the former royal’s toe made frontpage news. The scandal angered Prince Philip, and Sarah was banned from the royal household immediately after.

Sarah being banned from Queen Elizabeth’s vacation home in Sandringham meant that she could no longer celebrate Christmas with Prince Andrew and their two daughters. And at a young age, Princess Beatrice and Princess Eugenie took notice of the sudden changes.

In the article that she wrote for The Independent in 1996, journalist Rosie Boycott shared details from her conversation with the Duchess of York. Fergie learned that her daughters asked their dad why their mom is no longer allowed inside the “big house.”

According to Boycott, there was a time when Fergie decided to accompany Princess Beatrice and Princess Eugenie to Sandringham to celebrate the holidays. But since she was banned from entering the household, she waited outside the gate “like a dog in a quarantine” while Prince Andrew entertained their daughters.

But despite the ban, Sarah maintained a close relationship with her mother-in-law. During a previous interview, the mom of two said that she and the Queen never lost contact with each other. Sarah also declared that she loves the Queen so much.

“The Queen remained astonishingly tolerant,” Boycott said. “Despite the appalling publicity her daughter-in-law had brought on them all, she turned up in the afternoon for a few private seasonal whiskeys. Everyone else in Sarah's life might have been throwing her to the wolves, but her mother-in-law offered kindness and support — as she still does.”

Royal Family Queen Elizabeth the Queen Mother poses 04 August 1989 outside her London Clarence House residence for photographers with Queen Elizabeth (R), Sarah Ferguson, Prince Charles, Princess Diana and other members of the Royal Family on her 89th birthday. JOHNNY EGGITT/AFP/Getty Images