If there were a 10% chance that AI could wipe out humanity, no responsible government would leave its development to companies racing to build it. Yet until recently, that seemed to be the case. The warning was all the more ominous because it was made by a researcher at Anthropic, the trillion-dollar AI company behind the Claude chatbot. The dangers of AI-enabled pandemics or attacks on nuclear systems are real. Countries need not agree on democracy or trade policy to accept this.
The US and China will hold, reportedly, their first bilateral AI-safety talks before a planned White House summit between Donald Trump and Xi Jinping. Despite their tech rivalry, neither Washington nor Beijing can make the most advanced AI safe on their own. A US-China settlement won’t be able to say how AI works everywhere. Countries deploying AI must help write the global rulebook. The odds on an extinction event are shortening. This week Anthropic said that it identified five cases in which attempts were made using its models to support biological weapons development. It banned the accounts. In one case, a platform sent requests rejected by Claude to a rival with weaker safeguards. AI safety, clearly, cannot be that of the least responsible model.
