Tory chairman Nadhim Zahawi is desperately clinging onto his job amid warnings that the row over his tax affairs has made his position "untenable".
Allies of the Cabinet Minister said he had no plans to quit despite paying a multi-million pound penalty to resolve a dispute with HMRC.
It has emerged that he made the settlement - estimated to be around £5million - when he was presiding over the UK's tax policy as Chancellor last summer.
A source close to Mr Zahawi did not answer questions on the penalty but said: "He is absolutely not resigning."
Mr Zahawi was spotted arriving at Tory HQ this morning, saying "Morning" to journalists waiting outside.
Labour said his position had become "untenable" and Rishi Sunak had questions to answer over what he knew when he appointed him to his Cabinet.
In a statement at the weekend, Mr Zahawi said he had made a "careless rather than deliberate error" related to shares in YouGov, the polling firm he co-founded.
He said he had settled the matter and his tax affairs were up to date when he was appointed as minister without portfolio and Tory chairman by Mr Sunak in October.
It has now come to light that the dispute was resolve between July and September last year and the total amount paid is about £5million, according to the BBC.
Mr Zahawi was Chancellor from July 5 to September 6 2022.
Shadow Minister Jonathan Reynolds said: "People want all of their politicians to abide by the rules, they want anyone in public life not to see paying tax as somehow an optional thing, that if you're caught out you have to go back and do.
"They want that probity up front, that accountability, that professionalism, that integrity. That hasn't been the case here."
He added: "The sums of money here are not small. If the reports are true the amount of income that hasn't been declared will be very, very large indeed, and I just don't think in any way it's tenable to continue in a high profile position in public life when you've not behaved in that way."
Former Downing Street communications chief Sir Craig Oliver said Mr Zahawi was "hanging on by a thread".
"I think he's in serious trouble, you cannot be Conservative Party chairman and not go out and face the media," he told BBC Radio 4's Today.
"The problem for Nadhim Zahawi at the moment is it doesn't all add up. Why did you take the job as chancellor when you were clearly in dispute with the HMRC and he is yet to come out with an answer that is satisfying or feels comfortable on that point."
Tory former minister Tim Loughton told BBC Radio 4's Westminster Hour: "I think it's fair to let him (Mr Zahawi) have his say and put his side of the case. It would have just been more helpful if he had done that rather more fully rather earlier on."
He added: "If there is more to it, then he will absolutely have to stand up and take the consequences and the Prime Minister, I'm sure, will take the appropriate action.
"But at the moment we don't know the whole story."
Mr Zahawi has been approached for comment.
On Saturday, he said: "As a senior politician I know that scrutiny and propriety are important parts of public life. Twenty-two years ago I co-founded a company called YouGov. I'm incredibly proud of what we achieved. It is an amazing business that has employed thousands of people and provides a world-beating service.
"When we set it up, I didn't have the money or the expertise to go it alone. So I asked my father to help. In the process, he took founder shares in the business in exchange for some capital and his invaluable guidance. Twenty one years later, when I was being appointed chancellor of the exchequer, questions were being raised about my tax affairs. I discussed this with the Cabinet Office at the time.
"Following discussions with HMRC, they agreed that my father was entitled to founder shares in YouGov, though they disagreed about the exact allocation. They concluded that this was a 'careless and not deliberate' error.
"So that I could focus on my life as a public servant, I chose to settle the matter and pay what they said was due, which was the right thing to do.
"Additionally, HMRC agreed with my accountants that I have never set up an offshore structure, including Balshore Investments, and that I am not the beneficiary of Balshore Investments. This matter was resolved prior to my appointments as chancellor of the duchy of Lancaster and subsequently chairman of the party I love so much. When I was appointed by the Prime Minister, all my tax affairs were up to date."
No10 said it had nothing to add to the statement and said the Prime Minister had confidence in him as Tory chairman.
