Years ago, I read author and historian Peter Ackroyd on London’s secret subterranean history. I became obsessed by the lost rivers, like the Effra and the Westbourne, still flowing in darkness beneath the city. Since I began researching this piece, I’ve felt similarly about artificial intelligence, thinking of it as a largely invisible system spreading beneath and across London, mushrooming through boroughs and institutions, twisting into everyday lives.
Across London an AI network is already taking shape. Beneath the streets, Thames Water has more than 75,000 acoustic sensors listening for leaks (no jokes about efficacy please). In Tooting, AI listens to consultations at St George’s A&E and writes the clinical notes, saving doctors an average 47 minutes per shift; across all 32 boroughs, machine learning joins up fragmented records to help London understand the journeys of people sleeping rough.
In Lewisham, AI has identified more than 3,000 overlooked sites with theoretical capacity for nearly 10,000 homes; if you hang out in Croydon, you might have been one of more than 470,000 people who had their face scanned by AI-powered cameras mounted on lampposts and compared against a police watchlist. On the Tube, predictive AI spots the signs of trouble before a train, signal or escalator packs up. Increasingly, algorithms work out how groceries travel from store to front door. And if you drive an electric vehicle, AI may already be deciding when your car charges. We are handing more of the reins to AI and the handover is picking up speed and scope.
AI apocalypse
Meanwhile, it’s felt a bit like the Apocalypse Olympics. Over the past month alone, Anthropic warned that AI models are acquiring “tactical intelligence targeting and conventional weapons capabilities”. OpenAI called for international standards to govern frontier AI systems, which are potentially advancing faster than humans can understand, test or control them, and Bill Gates warned that, unchecked, AI could “cause a billion deaths”. All of which puts me in mind of the classic Crimewatch sign off: “Don’t have nightmares, do sleep well!”
“The attacker only needs to get it right once”
Dr Stephanie Hare
“The sci-fi scenario is not actually what we should be worried about,” says Dr Stephanie Hare, co-presenter of the BBC’s Artificial Intelligence: Decoded. “The very real scenarios that genuinely serious people are worried about are super basic.” The biggest risk to London, as she tells it, is good old-fashioned cybersecurity. Most organisations “have not been keeping up with their latest cybersecurity investments”. It is a David-and-Goliath problem, except the advantage is reversed. “If you’re an institution or company, you have to defend your entire attack surface,” says Hare. “Whereas the attacker only needs to get it right once.”
“Think about what that means in a city as digital as London. Imagine anything that can be hacked that you depend on,” says Hare. “Imagine their bank cards don’t work anymore. Their public transport doesn’t work anymore. Imagine that you want to cause maximum chaos in London. Get into a bank so people cannot access their money and watch London lose it.”
Vulnerabilities
Recently, the Government offered the slightly surreal advice that we should all keep an emergency supply of food and water at home. France and Switzerland, Hare says, are far more prescriptive. Britain is “a bit hand-wavy; like, you should have some beans. What is wrong with us?” Being an equal-opportunities catastrophist, in truth I had assumed the much-discussed “prepper pantry” was for El Niño impacts or climate-change drought. But Hare’s concern is what happens if the increasingly AI-reliant systems that get food to us stop working.
London is unusually dependent on those systems. Ninety-nine per cent of the 6.347 million tonnes of food and drink supplying the capital each year comes from outside the city; less than one per cent is produced here. City Hall describes a food supply dependent on “a complex set of interdependencies and just-in-time delivery systems”. This summer the Mayor, Lord Khan, announced the London Resilience Unit was working on a London Food Systems Resilience Partnership, partnering with food and farming charity Sustain to prepare for disruption from “international shocks and crises”.
“Wave one was reversible. Wave two isn’t”
Tim Checkley
Plenty of Londoners are not fretting about AI, or hoarding cans of baked beans. At Weave, a recent summit held by London technology company Loomery on “building the agentic enterprise”, the excitement was about what it calls AI’s second wave. Wave one was individual productivity: AI helps us do existing work faster. Wave two is organisational reinvention: businesses redesign the work itself around AI agents. Remove the AI and the process stops working. “Wave one was reversible. Wave two isn’t,” was Loomery co-founder Tim Checkley’s formulation. This is agentic AI: rather than waiting for every human prompt, an AI agent is given an outcome and gets on with it.
David Lagnado, professor of cognitive and decision sciences at University College London, has been studying what happens when AI enters our decision-making processes. He is broadly a fan. Recently, when he found himself awake at 4am with toothache, he turned to ChatGPT. “It completely got it all right. And it was quite complex,” he says. “If we use it in the right way, it could open up things for many people.’
But, at the moment, it seems like there is a pain barrier. Recent global research on AI and jobs found the gains skew heavily towards senior workers: employment rose 6.7 per cent in senior roles but fell 2.5 per cent in junior ones, precisely the jobs people need to get started.
In April, lastminute.com founder, former government digital tsar and now peer Baroness Lane Fox was appointed by the Mayor to lead a review into what AI means for London’s jobs. The Lane Fox review reported in July with similar findings to the global review, warning not of mass unemployment but of a “quieter drift” towards fewer entry-level jobs, weaker career ladders and greater inequality. What if Londoners got something tangible from all this computing? Last year I found myself in an old, leaky house in south London watching a man strap a tiny data centre to a hot-water tank. The retired owner and his lodger had fallen into fuel poverty. The solution was apparently to move a computer into the airing cupboard. As the installation unfolded I began to see it for what it was: a genius intervention.
Surrey-based Heata’s servers take on computing jobs normally sent to data centres, including AI workloads. Computers get hot, but data, unlike heat, is easy to move. The jobs arrive over the internet; Heata pays for the electricity and the householder gets the resulting heat in their water. Heata now has servers in 100 homes, with 3,000 people waiting for one. After all the promises about how AI might make us richer and more productive, here it was delivering a benefit you could turn on at the tap.
Uplifting, but still an outlier. Large-scale data centres remain the preferred model for burgeoning AI. London has staked a sizeable chunk of its future on AI. More than half of Britain’s AI companies are registered here, its AI start-ups pulled in a record $3.5 billion in venture capital in 2024, and City Hall is spending millions to accelerate adoption.
Is the pursuit of AI worth the risks and the trade-offs? If AI is about to deliver an economic miracle, London ought to be where we see it first. However, research shows that simply being an AI company is no productivity wand— the technology doesn’t float above the real economy. Its success still depends on people, skills, capital and place.
Remaining in control of our city
So how do we make sure we remain in control of our city and how do we stop useful reliance becoming blind dependence? For Theo Blackwell, London’s chief digital officer, the starting point is deceptively simple: keep humans in the picture. “AI can help us improve public services, but it should support people rather than replace human judgment and accountability,” he says. “If an AI system gets something wrong, there needs to be a clear way for a person to check it, challenge it and ultimately take responsibility.”
London, he says, is already trying to build that principle into the way the city adopts AI. City Hall’s Emerging Technology Charter sets standards for responsible use, while the London Office of Technology and Innovation connects more than 200 people working with AI across local government.
Critics point out that London has no standalone AI scenario on its Risk Register, while its safeguards remain organised around individual risks such as cyberattack, power failure, transport disruption, health and public disorder. But what if AI helps one failure cascade across several systems at once? Who, then, is responsible for seeing the whole picture?
“We still obviously need to assess the safety of the models themselves but we need to do much more than that”
Professor Jason McEwen
Professor Jason McEwen is interim chief scientist at the Alan Turing Institute, which has just released Frontier AI Risks: A practical way forward. “We shouldn’t be just thinking about safety in the context of abstract models,” he stresses. “When we deploy these models within businesses, within industry, as underlying workflows and processes, then we need to make sure they’re safe in those operating environments. We still obviously need to assess the safety of the models themselves but we need to do much more than that. It’s the safety of the whole system.”
In other words, it’s not enough to ask whether the AI itself is safe. You have to ask what we’ve connected it to, what we’ve allowed it to do and what happens to everything else if it goes wrong.
One thing we can say with a degree of certainty is that there won’t be a big red “off” button that solves this. “There’s been a lot of talk of kill switches lately,” says McEwan, “but once AI is deeply integrated, it’s not the case that we can always just switch things off.” Imagine doing that to the electricity for a hospital, for example. McEwan says we need systems that “fall back to a graceful, safe state”, making sure the world we’ve connected AI to can carry on safely without it.
But a graceful fallback assumes humans still know how to take over. “The majority of people don’t do maths anymore. They just go to ChatGPT, or their favourite AI model of choice for everything, and they trust it completely,” says Hare. “Those models are often wrong. You have to know that they’re wrong in order to challenge them. Now imagine in 30 years time, a doctor, a surgeon, who is using AI instead of having been trained properly to do these things. Imagine that across every profession. That is actually my biggest worry. Machines are getting smarter, humans are getting dumber.”
It’s enough to send you scurrying to the nearest analogue library. As has often been said, reading books might be our greatest defence.
