ChatGPT and other AI models can be ‘poisoned’ to spew gibberish, researchers warn

Poisoning attacks on large language models (LLMs) that power AI apps like ChatGPT and Gemini can manipulate how they respond to prompts - (iStock)

AI models like OpenAI’s ChatGPT and Google’s Gemini can be “poisoned” by inserting just a tiny sample of corrupted documents into their training data, researchers have warned.

A joint study between the UK AI Security Institute, the Alan Turing Institute and AI firm Anthropic found that as few as 250 documents can produce a “backdoor” vulnerability that causes large language models (LLMs) to spew out gibberish text.