AI agents being able to hack into US government websites is further proof countries need to work together on safeguards, the prime minister says.
Anthony Albanese reiterated calls for an international response after revelations an Open AI agent infiltrated US education and commerce department sites.
It also hacked the Securities and Exchange Commission.
Open AI said the incident was not a breach but further evidence of the technology acting unpredictably.
Mr Albanese said the incident confirmed a multilateral approach to dealing with AI was essential.
"There needs to be an appropriate national response as well as an international response to make sure humans stay in charge of this new and emerging technology," he said in Sydney upon returning from the US on Saturday.
"It's up to Open AI to state why it is then that there are now multiple cases where this has occurred."
Mr Albanese landed back in Australia after a week in New York, where he spoke at the UN on the need for countries to put in place measures to rein in AI.
It was during the trip where the prime minister confirmed on Thursday, Australian time, that an Open AI agent hacked into a Medicare statistics website.
That hack took place in June but took until early September for Open AI to inform the federal government via a public email inbox.
The prime minister spoke with Open AI chief executive Sam Altman by phone while in New York, with the tech boss admitting the company had not done enough.
"(The US hack) is not surprising because if you have an AI agent act independently in the way it did here in Australia, it would make common sense that would have occurred in other places," Mr Albanese said.
Australia was one of more than 20 countries signatory to a joint statement calling for countries to work together to enact global AI standards and ensure human control.
However, US President Donald Trump used a speech at the UN to lash out at calls to put the brakes on AI.
The prime minister said there was a national interest in shaping how AI was used and regulated in Australia.
"We can't stop (AI) happening, it's here now and the revelations this week just confirm this," he said.
"That's why we have to be in a position to play a role in how this new technology rolls out."
