OpenAI and Anthropic are pushing for their most advanced artificial intelligence systems to be independently tested before deployment, yet questions remain over how much authority government evaluators should have, leaving a gap over who will actually police the technology they say could be dangerous.
The two companies' chief executives have warned in essays, social media posts and even a speech to the United Nations that their most capable models could pose a threat to humanity and should not be deployed without independent safety checks.
The companies are also developing their own evaluation frameworks and working with outside evaluators, while government testing is already taking place through the US Center for AI Standards and Innovation, or CAISI.
Timing of AI Safety Push Raises Questions
The push comes as both companies court investors ahead of anticipated stock market listings, and as AI becomes a fixture of campaign messaging before the US midterm elections.
Sarah Shoker, a former OpenAI geopolitics lead now at the University of California, Berkeley's Risk & Security Lab, told AP that the emphasis on hypothetical 'superhuman' risk is not accidental.
She said it draws attention away from harms already occurring, including the use of AI in military technology that, in her words, is already being used to kill people. The debate intensified this month when Anthropic engineer Jacob Coxon resigned, calling publicly for a pause on development of advanced systems.
An Anthropic spokesperson said the company has called for regulation 'for several years'. An OpenAI spokesperson, Liz Bourgeois, said the company has paused training of its most advanced models, adding: 'People want to know AI is being developed safely, and that starts with what companies like ours do ourselves.'
President Donald Trump has dismissed warnings about AI risk as a 'HOAX' designed to benefit China, and has rejected the need for new AI regulations. Venture capitalist David Sacks, who co-chairs Trump's Council of Advisors on Science and Technology, has dismissed those urging caution as a 'Doomer Industrial Complex'.
No Agreed Standards for AI Safety Testing
A federal AI evaluation bosy already exists in the form of CAISI, renamed in 2025 from the Biden-era AI Safety Institute, which was established in 2023. CAISI is part of the National Institute of Standards and Technology and works on AI testing, standards and collaborative research.
But a parallel network of independent evaluators, from the nonprofit METR to commercial audit firms, has emerged alongside it. Andrew Strait, who recently left the UK's AI Security Institute, said that unlike aviation or financial services, there are still no universal standards for testing AI safety.
Conrad Stosz, former head of the US standards centre and now governance lead at evaluation lab Transluce, said it remains unclear what 'embedded' oversight would mean in practice.
Anthropic says it already submits systems for pre-deployment evaluation with CAISI and has worked with independent organisations including METR, while OpenAI has also published details of CAISI pre-deployment testing.
PitchBook analyst Harrison Rolfes said the strategy could also serve a commercial purpose, positioning the two firms as attractive options for investors and partners such as Nvidia and Google. 'They're creating a wall or a moat within this sector,' he said. 'It's genius and they're all going to make a lot of money.'
Not everyone agrees the alarm is warranted. Nvidia chief executive Jensen Huang said in a call with Trump that concerns about AI had become excessive.
But former OpenAI researcher Daniel Kokotajlo, who resigned in 2024 over similar concerns to Coxon's, said the current rhetoric risks serving to 'dissipate and redirect' pressure for regulatory action. He continues to express fears over AI's potential role in bioweapons and nuclear escalation.
The debate now centres on how independent AI testing should work and how much authority government evaluators should have as the technology advances.
