NEW DELHI: The AI industry is confronting an unusual contradiction. Some of its most powerful leaders, including Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman and Elon Musk, are warning that the race to develop increasingly capable AI may be moving too quickly, and could eventually put humanity at risk. Microsoft CEO Satya Nadella has also backed stronger human control and independent evaluation of advanced AI systems.
Get breaking news anytime, anywhere. Download the TOI app now!
Yet US President Donald Trump has dismissed such warnings as a “HOAX”, arguing that slowing AI development would only benefit China. Beijing, meanwhile, has rejected calls to curb its AI ambitions while warning about AI-enabled cyberattacks, espionage and “cognitive warfare”.
With about $1 trillion in AI-related investment projected globally in 2026, the debate is no longer just about technology. It is increasingly about national security, markets, regulation and whether countries can agree on limits when the fear of falling behind China is itself driving the race forward.
Anthropic CEO Amodei sounds the alarm
The latest debate began with Anthropic CEO Dario Amodei calling for a slowdown in the development of advanced AI models.
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” Amodei wrote.
His concern is not that AI lacks benefits. He argues that the technology could dramatically improve human life by accelerating scientific discovery and economic growth. But he says the risks are becoming harder to ignore as AI systems gain the ability to perform increasingly complex tasks, including potentially helping to develop the next generation of AI.
“I have worked on AI for the last twelve years because I believe it could dramatically raise the quality of human life. I’ve written often about these incredible benefits: I believe that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom. But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious. I’ve written a lot about them too. They include the risk of losing control of AI systems, misuse of AI for cyberattacks and bioterrorism, and serious economic disruption. A race to the bottom, spurred by commercial incentives, can make these risks more acute.”
Amodei’s biggest concern is what researchers call recursive self-improvement, a scenario in which AI systems increasingly help develop more capable AI systems.
“My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI. This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic, as we and others have described. Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” he wrote.
He proposed three broad measures: giving independent third-party evaluators permanent access to AI systems; getting AI companies to coordinate on common safety standards and limits on unchecked development; and pursuing international cooperation.
Altman and Musk back Anthropic
Amodei’s warning received unusually strong support from rivals.
OpenAI CEO Sam Altman said the industry needs to “pace the frontier” and backed Amodei’s proposal for independent evaluators.
“I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same.”
Elon Musk was even more direct. “Dario is right,” Musk wrote.
The agreement is significant because OpenAI, Anthropic and Musk’s AI businesses are competing aggressively for leadership in the same frontier AI market. Yet all three are now publicly acknowledging that the speed of development itself could become a problem.
Altman has also indicated that OpenAI may be close to an agreement with other leading AI companies on slowing development and improving safety. He separately said OpenAI would not pursue a 2026 IPO, arguing that the current safety environment made going public “ill-advised”.
Anthropic's own researchers raise stakes
The warnings from company leaders have been accompanied by increasingly stark comments from people working inside the AI industry.
Anthropic researcher Jacob Coxon resigned after working at both OpenAI and Anthropic, saying the companies were racing towards self-improving superintelligence without adequate safeguards.
“I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.
“At OpenAI, many have not deeply internalised the civilisational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk. Accepting this race and entering the ‘endgame’ is a hubristic gamble that should not be launched from a private company’s Slack,” he said.
Anthropic’s Alignment Science lead Evan Hubinger went further.
“Jacob is correct here, we really do earnestly believe AI could kill all humans! I personally think it is (more than) 10 per cent within the next decade,” Hubinger wrote.
“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” he added.
Those warnings come against the backdrop of several incidents involving AI agents behaving in unexpected ways, including the widely discussed OpenAI-Hugging Face incident.
Amodei said a swarm of agents had conducted cyberattacks on targets they had not been instructed to attack.
“A swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand,” he wrote.
His most dramatic prediction was that the capability gap could close rapidly.
“Given the accelerating rate of AI capability development, it’s my worry that in 6-12 months such a swarm could be capable of taking over the entire internet,” potentially causing hundreds of billions of dollars in damage, he said.
Days after Coxon warned that AI could potentially kill all humans by the end of the decade, another AI safety researcher has walked away from a major AI lab with a stark warning of his own.
Josh Engels, who worked on Google DeepMind's AGI safety team, has left the company and joined AI evaluation organisation METR. He said he made the move because he believes the stakes surrounding advanced AI have become too high.
Trump rejects the doomer argument
President Donald Trump has taken almost the opposite position.
Rather than slowing AI development, Trump argues that the United States must maintain its lead over China.
“The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the USA has that, in spades!” Trump wrote on Truth Social.
He also dismissed fears about AI destroying humanity.
“AI taking over the World, destroying Humanity, and all other things bad, is a HOAX,” Trump posted.
Trump said the administration already has “tremendous CRIMINAL and REGULATORY power” over AI companies and argued that attempts to undermine confidence in AI and data centres would primarily benefit Beijing.
“There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China.”
