Lord Cameron has resigned from the shadow cabinet along with Conservative party chairman Richard Holden, the party has announced.
Former prime minister Rishi Sunak will remain the leader of the Conservatives' interim shadow cabinet ahead of a leadership contest.
Andrew Mitchell will replace David Cameron as shadow foreign secretary.
The party has named Jeremy Hunt as shadow chancellor, with James Cleverly shadow home secretary, mirroring the portfolios they held in government.
It comes just three days after the Labour party swept to power following a landslide victory in the General Election, leaving the Tories with just 121 seats.
Mr Holden narrowly won the seat of Basildon and Billericay after being parachuted into the constituency at the eleventh hour, sparking fury from local Tories.
Kemi Badenoch has become shadow levelling-up secretary, while Kevin Hollinrake will shadow Ms Badenoch's former Business Secretary post and Mims Davies becomes shadow women and equalities minister.
Mr Fuller said in a statement: "The Conservative Party has had a difficult election and it is important that we regroup and reflect on these results. We should also challenge ourselves candidly and deeply on the strengths of the Conservative Party across the country and outline where improvements can be made.
"I am honoured to be asked to act as interim chairman of the Conservative Party and to be working alongside colleagues in the shadow cabinet.
"United as a party we will be ready and able to hold this new Labour government to account every step of the way."
Several former Cabinet ministers are expected to run for the Tory leadership, including ex-Health Secretary Victoria Atkins and former Home Secretary Suella Braverman.
In an article for the Sunday Telegraph, Mrs Braverman said the party needed to take a tougher line on reducing immigration, claiming some Tories had treated voters like “mugs” and Mr Sunak had run an “idiotic strategy”.
She said the party was facing an “existential threat” from Nigel Farage’s Reform UK.
Ms Atkins told the BBC that the country was still “instinctively Conservative” despite the worst election result in the party’s history.
Several senior Tories lost their seats in last week's poll.
Among them were former defence secretary Grant Shapps and former Commons leader Penny Mordaunt, who lost their seats to Labour candidates in Welwyn Hatfield and Portsmouth North.
Ex-education secretary Gillian Keegan and former justice secretary Alex Chalk also lost their seats in Chichester and Cheltenham to the Liberal Democrats.
