Google’s chatbot Bard got into difficulties when asked a question about space
(Picture: Google)The share price of Google owner Alphabet plummeted on Wednesday after it was revealed the upcoming Bardchatbot may not be as smart as Google would like us to think.
Around $100bn (£82bn) was swiped off Alphabet’s valuation yesterday after Bard gave out false information in Google’s own initial demo of the chatbot. At the time of writing the share price has only partially recovered.
The same shaky information is also seen in the company’s initial blog post about Bard, published on February 6.
Is Google Bard unsafe?
Bard claimed the James Webb Telescope produced the “the very first image of a planet outside our solar system”.
As multiple experts on Twitter pointed out, including Grant Tremblay from the Center for Astrophysics at Harvard & Smithsonian, that honour actually belongs to the NACO Very Large Telescope (VTL). It captured an image of a brown dwarf in 2004, 17 years before the James Webb Telescope launched.
As if to compound the impact of this blunder, the Google Bard demo shows someone asking for James Webb Telescope facts they can tell their nine-year-old child.
“This highlights the importance of a rigorous testing process, something that we’re kicking off this week with our trusted tester program,” said a Google spokesperson.
“We’ll combine external feedback with our own internal testing to make sure Bard’s responses meet a high bar for quality, safety and groundedness in real-world information.”
However, this may require more work than is possible in the few weeks Google has before it promised to release the bot to the public.
What are chatbot hallucinations?
Chatbots making up or cludging information is an effect common enough to have earned its own term, a “hallucination”.
“Any large language model that’s given an input or a prompt – it’s sort of not a choice – it is going to hallucinate,” Peter Relan of AI firm Got It AI told Datanami in January.
He estimates the rate of Bard rival ChatGPT’s hallucinations at 15% to 20% – up to a fifth of what these chatbots say may be faulty or false.
We noticed this effect when researching our battle of the chatbots article. We asked You.com’s YouChat bot about a company acquisition and it mixed up the owner of a company with the group buying it.
Chatbots can often fumble information in the sources they mine, or incorrectly join the dots between these pieces of information. Part of this is down to the way these AI bots are designed to seem “real” more than “right”. Google CEO Sundar Pichai describes Bard as a “experimental conversational AI service”.
Google and Microsoft have plans to integrate elements of this conversational AI into their web search software, potentially opening us up to a fresh hell of misinformation. This is not a new problem. AI hallucination was investigated in a 2018 Wired article as a issue that needed a fix, and that fix clearly has not been found yet.
