Removal vans were seen in Downing Street on Monday morning as Boris Johnson prepares to move out of Number 10.
The ousted Prime Minister and his wife Carrie are reportedly eyeing a move to Dulwich Village in south London with their two children.
The affluent area in Southwark sees average house prices of just over £1.8million, according to Rightmove.
Mr Johnson, the MP for Uxbridge and South Ruislip, is said to be looking for a home with a garden “away from the glare of central London”.
“Now they have Wilfred and Romy, a family home with a large garden is a must,” a source told the Sun. “That’s exactly what Dulwich can offer.”
Margaret Thatcher and her husband Dennis also chose the leafy neighbourhood after she was ousted as PM in 1990.
They stayed for a year before moving to a more central location.
Mr and Mrs Johnson are not overseeing the move this week as they are on holiday in Greece.
The couple spent at least £112,000 refurbishing the Number 10 flat when they moved in in 2019.
High-end designer Lulu Lytle was put in charge of the redecoration, which included installing gold-coloured wallpaper and fittings.
The work was initially paid for by the Cabinet Office, but £52,000 was given to the Tory party by Lord Brownlow to cover the decorating bills.
Mr Johnson later said he paid out of his own pocket, but the Conservative Party was fined £17,800 when the Electoral Commission found it had failed to accurately declare all of Lord Brownlow's donations.
A £30,000 annual grant, funded by the taxpayer, is available to Prime Minister's for upgrading their Downing Street home.
The PM's official spokesman said on Monday that Mr Johnson will be allowed to take furnishings that were not paid for by the tax payer to his new residence.
Asked whether the Prime Minister has any engagements this week, they added: "The Prime Minister is on holiday this week.
"As you know by virtue of their role they'll obviously be kept informed on any urgent issues and make decisions particularly those (related to) national security."
After returning from holiday, Mr Johnson is expected to work between Number 10 and his country mansion Chequers until September 5.
Before moving into Number 10 in 2019, Mr and Mrs Johnson bought a townhouse in Camberwell, near Brixton.
The home is now listed for sale at £1.6million, £400,000 more than they paid for it.
