AI chatbots may struggle with hallucinating made-up information, but new research has shown they might be useful for pushing back against unfounded and hallucinatory ideas in human minds. MIT Sloan and Cornell University scientists have published a paper in Science claiming that conversing with a chatbot powered by a large language model (LLM) reduces belief in conspiracies by about 20%.
To see how an AI chatbot might affect conspiratorial thinking, the scientist arranged for 2,190 participants to discuss conspiracy theories with a chatbot running OpenAI's GPT-4 Turbo model. Participants were asked to describe a conspiracy theory they found credible, including the reasons and evidence they believed supported it. The chatbot, prompted to be persuasive, provided responses tailored to these details. As they talked to the chatbot, it provided tailored counterarguments based on the participants' input. The study fielded the perennial AI hallucination issue with a professional fact-checker evaluating 128 claims made by the chatbot during the study. The claims were 99.2% accurate, which the researchers said was thanks to extensive online documentation of conspiracy theories represented in the model's training data.
