Services Australia failed to promptly report a security breach that allowed OpenAI to hack into a government database because it was sent to an email staff did not monitor closely.
Announcing a new taskforce to probe the breach, Government Services Minister Katy Gallagher said the delay in notifying authorities would form part of a "forensic investigation" of the hack, which Prime Minister Anthony Albanese revealed on the sidelines of the United Nations General Assembly.
Acting Prime Minister Richard Marles said the AI agent had, without authorisation, "climbed the fence" to access a database that sat behind a public-facing website, which held aggregated Medicare data used by academic researchers, after hitting a roadblock while the model was being trained.
The "very serious" incident on June 18 raised questions about how government websites could be secured against potential future hacks, he said.
The taskforce within the Department of Prime Minister and Cabinet will probe whether existing legislative, governance and information-sharing arrangements "are fit-for purpose to prepare for, and respond to, a cyber incident involving AI" and inform how to "build and maintain resilient systems in the AI era."
It will consider mechanisms to "strengthen networks and systems of government departments and agencies against AI vulnerabilities," along with potential new laws and penalties.
Agencies are required to notify the Australian Cyber Security Centre, which sits within the Australian Signals Directorate, of all cyber security breaches within 72 hours.
Senator Gallagher said when asked how it had taken Services Australia until September 15 - five days after OpenAI emailed the agency - to notify ASD, that it had taken "a couple of days to verify that what they'd been alerted to in the email was legitimate."
She said Services Australia had opened the email on September 11 and that the five-day period had included a weekend.
The agency still did not have all the information and would seek further details from OpenAI, whose chief executive Sam Altman met with Mr Albanese on Wednesday.
Senator Gallagher, who only heard about the incident on September 17, said OpenAI had disclosed the breach by emailing [email protected], which was "looked at once a day."
Its inbox received "quite a number of notifications" and "many of them are hoaxes," she said.
Services Australia and ASD had "made very clear" in a meeting with OpenAI last week that it should have used direct communication channels to alert the breach, rather than a "generic" email address.
Mr Marles said it was "not good enough that that's how we first become notified of it."
ACT independent senator David Pocock, who has been critical of the government's decision to shelve plans for a standalone AI Safety Act, said the OpenAI breach was "very concerning but ultimately unsurprising" and "highlights how slow the Australian Government has been to implement appropriate safeguards."
Community and Public Sector Union national secretary Rebecca Fawcett, who is pushing for stronger AI protections as part of APS-wide bargaining, said "clear and mandatory safeguards" were needed.
The acting Prime Minister and Senator Gallagher addressed reporters in Sydney on Thursday, after Mr Albanese announced the breach at a press conference in New York overnight.
Senator Gallagher said she was considering if measures funded in the May budget with $160 million over four years for system upgrades to improve cyber security at Services Australia, "can or should be accelerated."
She said the government's other "legacy public-facing websites" could be decommissioned, with their data either moved to the more secure data.gov.au or an another secure platform.
Mr Marles said that, while no personal information about individuals had been accessed, "this is a warning about the technology being developed without safeguards and without guardrails in place."
The taskforce will work with the National Cyber Security Coordinator, the Australian Signals Directorate, the Australian AI Safety Institute and Services Australia.
It will examine reporting requirements relating to AI-driven cyber incidents, AI-identified cyber vulnerabilities and cyber-related AI safety incidents, including reporting obligations, thresholds, pathways and systems.
Commonwealth governance and information-sharing arrangements for managing incidents involving AI, including roles and responsibilities and escalation pathways, will also be reviewed, along with engagement and information-sharing obligations of AI firms, including notification and cooperation requirements.
And the review will consider the "adequacy of existing system and legislative, regulatory and enforcement frameworks in deterring AI-driven cyber incidents, including whether current offences, liabilities, penalties and enforcement mechanisms are sufficient and effective."
Mr Marles said that while the government sought to be "constantly improving in an emerging environment," Australia had a world-leading approach to cyber security in government systems and that databases containing sensitive or classified information had stronger protections.
"AI is a transformational technology. It offers so much better benefit for humanity, but it comes with danger," he said.
"It is really important that the safeguards and the guardrails are well ahead of the capability itself. And that is very much the position of the Australian government and the way in which we will be engaging with this technology and seeking to regulate and manage it in an Australian context."
