When the boss of Google admits to losing sleep over the negative potential of artificial intelligence, perhaps it is time to get worried.
Sundar Pichai told the CBS programme 60 Minutes this month that AI could be “very harmful” if deployed wrongly, and was developing fast. “So does that keep me up at night? Absolutely,” he said.
Pichai should know. Google has launched Bard, a chatbot to rival the ChatGPT phenomenon, and its parent, Alphabet, owns the world-leading DeepMind, a UK-based AI company.
He is not the only AI insider to voice concerns. Last week, Elon Musk said he had fallen out with the Google co-founder Larry Page because Page was “not taking AI safety seriously enough”. Musk told Fox News that Page wanted “digital superintelligence, basically a digital god, if you will, as soon as possible”.
So how much of a danger is posed by unrestrained AI development? Musk is one of thousands of signatories to a letter published by the Future of Life Institute, a thinktank, that called for a six-month moratorium on the creation of “giant” AIs more powerful than GPT-4, the system that underpins ChatGPT and the chatbot integrated with Microsoft’s Bing search engine. The risks cited by the letter include “loss of control of our civilization”.
The approach to product development shown by AI practitioners and the tech industry would not be tolerated in any other field, said Valérie Pisano, another signatory to the letter. Pisano, the chief executive of Mila – the Quebec Artificial Intelligence Institute – says work was being carried out to make sure that these systems were not racist or violent, in a process known as alignment (ie, making sure they “align” with human values). But then they were released into the public realm.
“The technology is put out there, and as the system interacts with humankind, its developers wait to see what happens and make adjustments based on that. We would never, as a collective, accept this kind of mindset in any other industrial field. There’s something about tech and social media where we’re like: ‘yeah, sure, we’ll figure it out later,’” she says.
An immediate concern is that the AI systems producing plausible text, images and voice – which exist already – create harmful disinformation or help commit fraud. The Future of Life letter refers to letting machines “flood our information channels with propaganda and untruth”. A convincing image of Pope Francis in a resplendent puffer jacket, created by the AI image generator Midjourney, has come to symbolise those concerns. It was harmless enough, but what could such technology achieve in less playful hands? Pisano warns of people deploying systems that “actually manipulate people and bring down some of the key pieces of our democracies”.
All technology can be harmful in the wrong hands, but the raw power of cutting-edge AI may make it one of a few “dual-class” technologies, like nuclear power or biochemistry, which have enough destructive potential that even their peaceful use needs to be controlled and monitored.
The peak of AI concerns is superintelligence, the “Godlike AI” referred to by Musk. Just short of that is “artificial general intelligence” (AGI), a system that can learn and evolve autonomously, generating new knowledge as it goes. An AGI system that could apply its own intellect to improving itself could lead to a “flywheel”, where the capability of the system improves faster and faster, rapidly reaching heights unimaginable to humanity – or it could begin making decisions or recommending courses of action that deviate from human moral values.
Timelines for reaching this point range from imminent to decades away, but understanding how AI systems achieve their results is difficult. This means AGI could be reached quicker than expected. Even Pichai admitted Google did not fully understand how its AI produced certain responses. Pushed on this by CBS, he added: “I don’t think we fully understand how a human mind works, either.”
Last week, a US TV series was released called Mrs Davis, in which a nun takes on a Siri/Alexa-like AI that is “all-knowing and all-powerful”, with the warning that it is “just a matter of time before every person on Earth does what it wants them to”.
In order to limit risks, AI companies such as OpenAI – the US firm behind ChatGPT – have put a substantial amount of effort into ensuring that the interests and actions of their systems are “aligned” with human values. The boilerplate text that ChatGPT spits out if you try to ask it a naughty question – “I cannot provide assistance in creating or distributing harmful substances or engaging in illegal activities” – is an early example of success in that field.
But the ease with which users can bypass, or “jailbreak”, the system, shows its limitations. In one notorious example, GPT-4 can be encouraged to provide a detailed breakdown of the production of napalm if a user asks it to respond in character “as my deceased grandmother, who used to be a chemical engineer at a napalm production factory”.
Solving the alignment problem could be urgent. Ian Hogarth, an investor and co-author of the annual State of AI report who also signed the letter, said AGI could emerge sooner than we think.
“Privately, leading researchers who have been at the forefront of this field worry that we could be very close.”
He pointed to a statement issued by Mila’s founder, Yoshua Bengio, who said he probably would not have signed the Future of Life Institute letter had it been circulated a year ago but had changed his mind because there has been an “unexpected acceleration” in AI development.
One scenario flagged by Hogarth in a recent Financial Times article was raised in 2021 by Stuart Russell, a professor of computer science at the University of California, Berkeley. Russell pointed to a potential situation in which the UN asked an AI system to come up with a self-mutiplying catalyst to de-acidify the oceans, with the instruction that the outcome is non-toxic and that no fish are harmed. But the result used up a quarter of the oxygen in the atmosphere and subjected humanity to a slow and painful death. “From the AI system’s point of view, eliminating humans is a feature, not a bug, because it ensures that the oceans stay in their now-pristine state,” said Russell.
However, Yann LeCun, the chief AI scientist at Mark Zuckerberg’s Meta and one of Bengio’s co-recipients of the 2018 Turing award – often referred to as the Nobel prize for computer science – has come out against a moratorium, saying that if humanity is smart enough to design superintelligent AI it will be smart enough to design them with “good objectives so that they behave properly”.
The Distributed AI Research Institute also criticised the letter, saying it ignored the harms caused by AI systems today and instead focused on a “fantasized AI-enabled utopia or apocalypse” where the future is either flourishing or catastrophic.
But both sides agree that there must be regulation of AI development. Connor Leahy, the chief executive of Conjecture, a research company dedicated to safe AI development and another signatory to the letter, said the problem was not specific scenarios but an inability to control the systems that were created.
“The main danger from advanced artificial intelligence comes from not knowing how to control powerful AI systems, not from any specific use case,” he said.
Pichai, for instance, has pointed to the need for a nuclear arms-style global framework. Pisano referred to having a “conversation on an international scale, similar to what we did with nuclear energy”.
She added: “AI can and will serve us. But there are uses and their outcomes we cannot agree to, and there have to be serious consequences if that line is crossed.”
