Sajid Javid has taken a veiled swipe at top Tories for admitting laws were broken at lockdown parties in Downing Street and Whitehall.
Scotland Yard revealed on Tuesday that 20 fixed penalty notices would be handed to people who attended law-breaking gatherings during the pandemic.
Boris Johnson and No10 have so far bizarrely refused to accept laws were broken - despite police saying fines would be issued for Covid law-breaking.
Two Cabinet Ministers have now publicly accepted that the fines meant people had broken the law over Partygate.
International Trade Secretary Anne-Marie Trevelyan said the Metropolitan Police's decision to issue the fines meant criminality had been discovered, echoing the view of Justice Secretary and deputy prime minister Dominic Raab.
But Mr Javid refused to answer questions about Partygate - and took a swipe at his colleagues, saying "it's never good practice for ministers to comment" on a live investigation.
Speaking to reporters at Barking Community Hospital in Essex, he said: "The police investigation... into events in Downing Street, it's a live investigation. It's still going on.
"I do know of course like everyone else that at least 20 fines have been issued so far but it is a live investigation. I think it's never good practice for ministers to comment on it."
Asked why there was so much "confusion" in Government about the matter, Mr Javid said: "I don't think there's any confusion.
"I mean, if you're asking me in general about fixed penalty notices, when those are issued of course it means the police issuing that has come to a conclusion, and that's purely a police matter - not a ministerial matter."
No10 again refused to acknowledge Covid laws had been broken.
The PM's official spokesman said: "The facts are not in dispute. We have been clear that mistakes were made, the Prime Minister believes it is right to respond once the full facts are known, once the investigation has concluded."
Keir Starmer weighed into the row on Thursday - and said the PM's wife Carrie Johnson should reveal if she receives a fine.
"My focus is on the Prime Minister because he is the one who sets the culture, he is the one who oversaw this criminality at his home and his office, he is the one that came to Parliament and said all rules were complied with, which is clearly not the case," he told reporters.
"So I do think Carrie Johnson should be named if she gets a penalty notice, but my focus is laser-like on the Prime Minister."
