If you're having trouble sleeping, there's no better resource than the experts. Board-certified sleep specialists can offer personalized and effective science-backed sleep guidance based on your unique health and lifestyle factors — if you can access a professional, that is.
However, many people are turning their backs on medical professionals for the convenience of AI. Research shows that 15% of the UK consult AI chatbots instead of a doctor, while a survey found that 49% of insured Americans have asked AI tools for medical advice. But can can a chatbot ever truly fix your insomnia?
To find out, I asked three of the most popular AI-generative assistants — Gemini, ChatGPT, and Copilot — the same request for sleep health advice. I then called on Mattress Firm's Dr Jade Wu, a board-certified sleep specialist, to evaluate and rate each response. Here's what she had to say and why AI's response might seem good at first but can't delve beneath the covers to provide long-lasting help.
My experience with Gemini, Copilot, and ChatGPT
To see whether AI chatbots can actually give sound sleep advice, I asked Gemini, Copilot, and ChatGPT the exact same question, prefaced with the prompt; "Act as a board-certified sleep doctor," to ensure each assistant knew I wanted it to replicate expert-level advice. Here's the full prompt:
"Act as a board-certified sleep doctor. I average 6 hours of sleep a night and have an erratic bedtime and wake-up schedule. In 3 actionable steps, advise me on how I can fall asleep fast, sleep through the night and increase my average sleep duration to 7 hours a night."
Each one delivered guidance in an instant and that split-second response set off alarm bells. But then I read the responses...
I frustratingly couldn't find any flaws. To me, each answer offered sound recommendations, with tips that seemed to come straight out of a Cognitive Behavioral Therapy for Insomnia (CBT-I) textbook (and, on reflection, quite probably did.)
"If you're awake in bed for roughly 20–30 minutes, get up and do something quiet in dim light until you feel sleepy again," said ChatGPT, bringing up the gold-standard sleep rule.
"Get 10–15 minutes of outdoor light within the first hour of waking — this accelerates melatonin shutoff and makes it easier to fall asleep at night," advised Copilot, displaying knowledge about our circadian rhythm and the morning sunlight rule (...that I already follow.)
"Waking up at the same time every day builds 'sleep drive' (the biological hunger for sleep) throughout the day," Gemini told me, promoting the sleep consistency that experts have always championed.
Still, some cracks started to show. For a start, I was being given advice I already followed, rather than being questioned on my routine.
Gemini was also the only AI that came with a flat-out disclaimer to remind me that it's not a doctor. (However, Copilot did mention that if experiencing persistent insomnia, loud snoring, or breathing pauses at night, it’s important to speak with a healthcare professional — but this caveat wasn't as prominent as Gemini.)
"This is for informational purposes only", warned Gemini at the beginning of its answer. "For medical advice or diagnosis, consult a professional."
And when I showed all three responses to Dr Wu, she highlighted all the ways that disclaimer rang true.
A board-certified sleep doctor rates sleep advice from 3 AI assistants
Is ChatGPT sleep advice trustworthy?
When evaluating ChatGPT's advice, the first thing Dr Wu noticed was its lack of a disclaimer.
"Instead, it just launched right into advice with 'I' sentences spoken from the perspective of a medical professional," the board-certified doctor pointed out. "I don't love that."
While Dr Wu did say that ChatGPT's sleep guidance was "mostly evidence-based and sound", she tells me that the problem lies in its generalizing.
"It did not ask any follow up questions to try to figure out what exactly the sleep problem is, making its advice generic and not necessarily relevant — it may even backfire," she warned.
Dr Wu's ChatGPT rating: 2 out of 5
Is Copilot and Gemini's sleep advice trustworthy?
First of all, Dr Wu is pleased that both Copilot and Gemini did come with the caveat that it's always best to speak to a healthcare professional, but both still suffered from one-size-fits-all responses.
"They're better than ChatGPT's rating because at least there are disclaimers, but it's not 5 because, again, it did not do any assessment to see what the sleep problem actually is and what other contextual factors might be at play," she says.
Follow up questions the sleep expert herself would ask include "Are there any other symptoms, like signs of sleep apnea?" and " What have they already tried, and how well did it work?" She would also delve into daytime habits (such as whether the person naps) and work schedules, and why my question is fixated on getting 7 hours of sleep if I feel rested.
"Depending on the person's age, lifestyle, genetics, and many other factors, 6 hours might be enough, or perhaps even 7 is not enough," Dr Wu says.
"And we're just scratching the surface...I haven't even gotten to co-morbid health conditions, social factors and family history," she adds.
Dr Wu's Copilot and ChatGPT rating: 3 out of 5
Did any of the AI assistants provide good sleep advice?
While all AI assistants presented oversimplified advice without taking into account personal factors such as age, genetics, and lifestyle, Dr Wu didn't have any major concerns about the advice itself at a surface level.
"I like that all the models started with anchoring the wake up time and stabilizing sleep-wake schedule," she tells me. "The morning light exposure and evening wind-down period are also very good."
"All of them provide 'okay' advice in the sense that none of it is egregiously wrong, and there are some reasonably universal tips that would be good for almost everyone, such as stabilizing wake timing if possible," she continues.
That said, she admits that she can't look past the lack of questioning. Depending on what the sleep problem is, the advice could be "very different, or even opposite," she says.
"None of them even begin to try to understand what the real sleep problem might be," the sleep specialist says. "And all of them take the person's prompt question at face value, without pushing back at all or questioning the assumptions in the question itself. "
Dr Wu's 3 tips on how to ask AI for health advice
1. Be specific with your questions
According to Dr Wu, most flaws in the AI assistant's advice are a result of a vague or uninformed question you've asked. That might mean you're missing the real issue with your sleep or getting advice you already follow (see Copilot telling me to go outside in the morning.)
But without sleep advisor training, it's hard to identify these gaps in your knowledge.
"The main thing to remember is that you don't know what you don't know," she explains. "And that's not your fault! If you're not a Board-certified sleep clinician, why would you understand sleep neuroscience or clinical treatment at a deep level?"
2. Don't let faulty assumptions guide your questioning
There are lots of assumptions around what 'healthy sleep' looks like. But Dr. Wu points out that sometimes these assumptions are wrong, and she uses my 'fall asleep fast' prompt to illustrate this.
"Falling asleep fast is not necessarily a good thing," she explains. "In fact, it may indicate being sleep deprived, having sleep apnea, or having another problem."
So before you start grilling ChatGPT, remember that a healthy sleep onset latency is 15 to 30 minutes and falling asleep as soon as your head hits the pillow is likely a sign of sleep deprivation. Be clear, keep to the facts and share relevant details to get better advice.
3. Make the AI model push back or ask questions
As mentioned above, Dr Wu says that follow-up questions are a sign of a good sleep advisor.
"If the model doesn't push back on your assumptions or doesn't ask a lot of good follow up questions to figure out your holistic sleep situation, then you're not going to get good advice," she warns. "You may even get advice that backfires or delays you from getting real sleep diagnostics done."
While the AI chatbot can give you an overview of general sleep hygiene tips, without drilling into your exact circumstances, it's never going to get to the root of your sleep problems. For that, it's worth the effort of finding a specialist.
