Anthony Albanese is confident other countries will add their voices to growing calls to rein in the rapid growth of artificial intelligence.
Australia was one of more than 20 countries that signed a joint statement urging additional safeguards for AI and to keep it under human control.
The statement was issued ahead of the United Nations General Assembly in New York, where AI measures are set to be on the agenda.
"AI must remain under human direction, oversight and control. It must be developed and used in line with international law," the statement, released on Tuesday, Australian time, said.
"Leading scientists and executives are warning that the pace of development could outpace our ability to manage emerging risks."
Alongside Australia, leaders such as Canadian Prime Minister Mark Carney, German Chancellor Friedrich Merz, Spanish Prime Minister Pedro Sanchez and European Commission president Ursula von der Leyen signed the statement.
The joint statement also remained open for endorsement for other countries to add to its list of signatories, something Mr Albanese hoped would happen.
"Other nations are putting forward their national interests," he told reporters in New York on Tuesday, Australian time.
"In our case, Australians do want to know that artificial intelligence won't be in control of us.
"We need to shape how it rolls out, rather than have artificial intelligence shape the way that our society functions."
The statement comes just days before US President Donald Trump and Chinese President Xi Jinping meet in Washington on Thursday, US time.
Mr Trump has labelled concerns about the rise of AI as a hoax, saying he did not want to cede ground in the technological race to China.
Mr Albanese said Australia played a central role in putting the statement together, with the prime minister having direct discussions with Finnish President Alexander Stubb, who issued the joint position on behalf of other nations.
"We continue to engage with our neighbours, this has been something that is discussed between leaders, just as it's been discussed around lounge rooms," he said.
"Artificial intelligence will have a massive impact on the way that our lives and our work is conducted."
The statement said risks with AI needed to be addressed, without widening the gap between countries for access to some of the benefits of the technology.
The signatories called for AI companies to develop transparent safety protocols and for governments to develop common standards for the technology.
It also urged the UN to explore options to create an international institution that could set out AI standards.
Mr Albanese met with Ms von der Leyen and European Council President Antonio Costa on the sidelines of the UN, where technology was also a focus.
The European Commission said Australia had been a trailblazer in implementing a ban on under 16s using social media, noting the EU was looking at similar laws.
A free-trade agreement with the EU was signed earlier in 2026, which coincided with a visit by Ms von der Leyen to Australia.
She also said she was pleased Australian researchers would soon be ale able to join Horizon Europe, the EU's funding program for innovation.
Mr Albanese said Australia joining the program represented a significant advancement.
"Australia's always been good at innovation and research and scientific advancement. We often have missed out on a bit of the commercialisation," he said.
Ahead of the UN General Assembly, Mr Albanese also gave an address to the Partners for Multilateralism, International Law, Peace and Prosperity summit.
It was attended by the European Commission president and Canadian Prime Minister Mark Carney.
Mr Albanese said countries could not afford to be isolated in turbulent periods.
"Global challenges cannot be solved by countries acting alone, especially right now, when we are facing one of the toughest periods in modern history," he said.
