Senior Tory Michael Gove took part in a “secret” cross-party meeting where the failings of Brexit were reportedly discussed.
The summit, held last week at Ditchley Park country house in Oxfordshire, was attended by high-ranking politicians, diplomats, business experts and Leave and Remain campaigners.
David Lammy and John Healey, two members of Keir Starmer's top team, attended the two-day gathering, according to the Observer.
The meeting was billed as a “private discussion” under the title: ‘How can we make Brexit work better with our neighbours in Europe?”
In an introductory statement, attendees heard there was now a view among “some at least, that so far the UK has not found its way forward outside the EU”.
A source told the newspaper: “The main thrust of it was that Britain is losing out, that Brexit is not delivering, our economy is in a weak position”.
It comes more than three years after the UK officially severed ties with Brussels following Boris Johnson's election promise to "Get Brexit Done".
But the UK's relationship with the EU is far from resolved, with efforts to resolve trading rules governing Northern Ireland currently deadlocked.
A Labour source played down the summit, telling The Mirror: “This was a bog standard Ditchley Park conference. Their events are always cross-party”.
Mr Gove, who was one of the key figures of the 2016 Vote Leave campaign, is understood to have attended the event in his capacity as governor of the Ditchley Foundation.
It came as the minister for the newly created science and technology department signalled the UK is “ready to go it alone” if the EU does not agree to Britain’s post-terms of membership of Horizon.
With progress stalled, Cabinet minister Michelle Donelan said she would “not sit idly by while our researchers are sidelined”.
"If we cannot associate, we are more than ready to go it alone with our own global-facing alternative, working with science powerhouses such as the US, Switzerland and Japan to deliver international science collaborations," she wrote in the Telegraph.
"The time for waiting is quickly coming to an end and I will not shy away from striking out alone."
But Royal Society President, Sir Adrian Smith insisted that her "first job" as Science Secretary "must be to secure association to Horizon Europe and other EU science programmes".
"These schemes support outstanding international collaboration and without being part of them we are undermining the Prime Minister's stated ambition for the UK to be at the forefront of science and technology globally," he said.