The U.K. government’s minister for artificial intelligence has warned that nations will need to “harden their defenses” against AI risk after a former Anthropic employee said that a human extinction-level event could happen in the next decade.
Answering a question from Fortune on the claim by former Anthropic worker, Jacob Coxon, Kanishka Narayan said he took the warnings seriously. Coxon warned in a post on X, which has been seen 175 million times, that “the people building AI earnestly believe that it could kill us all by the end of the decade.”
Anthropic scientist, Evan Hubinger, agreed with Coxon, saying the risk was greater than 10%. At the weekend Bill Gates, the founder of Microsoft, told NBC’s Meet the Press that, in the wrong hands, AI was “certainly powerful enough to drive events that, you know, cause a billion deaths.”
“The first thing to say is that when people in labs tell you that they’re very worried about this, you have to take that seriously,” Narayan said at the annual conference of the Labour Party, the governing party in the U.K. “You have to harden and build your defenses. The risks have only grown, so testing is good but clearly insufficient.
“We want to lead the world in being able to convene [countries] from the G20 next year as part of our presidency. Of course, we want to be fundamental drivers of the architecture of principles that are going to hopefully come out of that.”
U.K. Prime Minister Andy Burnham told the United Nations General Assembly last week that AI frontier labs were a “multiplier of risk” and that the U.K. could play the role of honest broker between competing national interests. Democrats and Republicans have called for new legislation aimed at regulating labs.
U.S. President Donald Trump, who said that talk of AI risks was a hoax, has warned that controlling the pace of US frontier lab research would hand China a significant advantage.
“The risks have only grown, so testing is good but clearly insufficient.”
Kaniskha Narayan, u.k. minister for artificial intelligence
“I have concerns that if we don’t win AI, we’re going to be put in a very bad position. We are leading China right now by a pretty good period,” Trump said. The White House is reported to have warned G20 members to take a “hands off” approach to new, global regulations.
Narayan, who sits in the Prime Minister’s Cabinet, the U.K.’s most powerful political body, said that there were a number of risks associated with AI.
“The candid sense I have is that I’m much more focused on risks today in terms of a secular uplift of cybersecurity risks that each of us in our organizations and lives,” he said. “[I am] concerned about the proliferation of these tools in a way that drives up biochemical risk as well, and yes, I am concerned about instances where models go beyond what humans have asked them to do as well.
“The sort of logical loops you have to jump through to make that an existential question are quite significant and severe, and so that’s an indication of how I think about the portfolio of risks here.
“You’ve got to do the best testing in the world to be able to understand the risks and mitigate them. We’re doing that. We’ve been doing that for three years. We’re very uniquely placed compared to any other country to be able to do that.”
Narayan also said there were many positives from AI, including in job creation, searching for jobs and productivity. He said that the UK had underplayed the role of innovation and technology IP in driving economic growth.
“We, as organizations, as individuals, as government, as the private sector, we actually have agency in whether this ends up being a technology that augments human labor, or is a technology that automates human labor,” he told a meeting organized by the Resolution Foundation, a think tank.
“There are ways of doing technological adoption that means more jobs and more productivity.”
