The UK’s biggest crisis text line for people needing urgent mental health support gave third-party researchers access to millions of messages from children and other vulnerable users despite a promise never to do so.
Shout, a helpline launched with a £3m investment from the Royal Foundation of the Duke and Duchess of Cambridge, says it offers a confidential service for people struggling to cope with issues such as suicidal thoughts, self-harm, abuse and bullying.
An FAQ section on its website had said that while “anonymised and aggregated”, “high-level data” from messages was passed to trusted academic partners for research to improve the service, “individual conversations cannot and will not ever be shared”.
But that promise was deleted from the site last year, and access to millions of messages from people – including children under 13 – has since been given to third-party researchers, the Observer has found.
Mental Health Innovations, the charity that runs the helpline, said all users agreed to terms of service that allowed data to be shared with researchers for studies that would “ultimately benefit those who use our service and the broader population”.
But the findings have led to a backlash among privacy experts, data ethicists and people who use the helpline, who said the data sharing raised ethical concerns.
More than 10.8m messages from 271,445 conversations between February 2018 and April 2020 were used in a project with Imperial College London aimed at creating tools that use artificial intelligence to predict behaviour, including for instance, suicidal thoughts.
Personal identifiers including names and phone numbers were removed before the messages were analysed, according to Shout, which says the messages had been “anonymised”.
But data used in the research included “full conversations” about people’s personal problems, according to the study.
Part of the research was aimed at gleaning personal information about the texters from their messages, such as their age, gender and disability, to get a better understanding of who was using the service.
Cori Crider, a lawyer and co-founder of Foxglove, an advocacy group for digital rights, said giving researchers access to the messages raised “serious ethical questions”. “Trust is everything in health – and in mental health most of all. When you start by saying in your FAQs that ‘individual conversations cannot and will not ever be shared’; and suddenly move to training AI on hundreds of thousands of ‘full conversations’, you’ve left behind the feelings and expectations of the vulnerable people you serve,” she said. “It’s particularly sad because this can set back proper consented research by years. Lack of trust in these systems can discourage desperate people from coming forward.”
Others raised concerns that people using the service could not truly comprehend how their messages would be used when at crisis point – even if they were told it was going to be used in research – because of their vulnerable state. About 40% of people texting the helpline are suicidal, according to Shout, and many are under 18.
Claire Maillet, 30, who used the service for support with anxiety and an eating disorder in 2019 and 2020, said she had no idea messages could be used for research. “My understanding was that the conversation would stay between me and the person I was messaging and wouldn’t go any further,” she said. “If I had known that it would, then I can’t say for certain that I still would have used the service.”
Maillet said she “absolutely understood” the need to conduct research in order to “make services better” in cases where people could give their informed consent.
But she said using conversations from a crisis helpline raised ethical concerns, even when names were removed, due to the nature of what’s being discussed.
“When you’re at that crisis point, you’re not thinking, ‘Will this information be used for research?’ You can spin it in a way that makes it sound good, but it feels like they are exploiting vulnerability in a way.”
Phil Booth, coordinator of medConfidential, which campaigns for the privacy of health data, said it was “unreasonable to expect a young person or any person in crisis to be able to understand” that their conversations could be used for purposes other than providing them support. “There are clearly reasons why Shout might need to retain data: no one is denying that. It’s these other uses: they are just not what people would expect.”
He added that it was “misleading” and “plainly wrong” that the charity had told users that their individual conversations would not be shared. “If you make a promise like that you should stick to it,” he said.
Shout was set up in 2019 as a legacy of Heads Together, a campaign led by the Duke and Duchess of Cambridge and Prince Harry aimed at ending stigma around mental health. The goal of the service is to provide round-the-clock mental health support to people in the UK, many of whom struggle to access overstretched NHS services, who can text a number and receive a response from a trained volunteer.
Mental Health Innovations states that a key goal is to use “anonymised, aggregated data to generate unique insights into the mental health of the UK population”.
“We use these insights to enhance our services, to inform the development of new resources and products, and to report on trends of interest for the broader mental health sector,” it says.
When service users first contact Shout, they are told: “This is confidential unless we’re worried for your safety and must tell someone to keep you safe.” They are also sent a link to a text with a link to the website and privacy policy with the message: “In using the service you confirm acceptance of our terms and consent to processing of your data.”
However, previous versions of the FAQs made a promise that has since been deleted, meaning people who texted the service in the past – including those whose messages were used in the research – are likely to have read information that was later changed.
An archived version of the helpline’s website from March 2020 shows that, in the FAQ section, users were told: “We anonymise and aggregate high-level data from conversations in order to study and improve our own performance. We share some of this data with carefully selected and screened academic partners, but the data is always anonymised, individual conversations cannot and will not ever be shared.”
In late spring 2021, the FAQ section was updated and the promise about individual conversations removed. The privacy policy was also updated to say that as well as sharing data, it could be used to “enable a deeper understanding of the mental health landscape in the UK more generally”, opening the door to a broader range of uses.
The Imperial study, published in December 2021, involved analysis of “entire conversations” manually and using artificial intelligence, according to the paper. To improve the service, it had three goals: predicting the conversation stages of messages, predicting behaviours from texters and volunteers and “classifying full conversations to extrapolate demographic information” – for instance by scanning messages for things such as numbers or mentions of school life, which might indicate a user’s age.
“Looking forward, we aim to train models to classify labels that could be considered directly actionable or provide a direct measure of conversation efficacy. For example, a key future aim is prediction of a texter’s risk of suicide,” wrote the researchers.
Data was handled in a trusted research environment and the study received ethics approval from Imperial, which said the research “fully complied” with “stringent ethical review policies” and provided “crucial insights”.
But Dr Sandra Leaton Gray, associate professor of education at University College London, an expert on the effects of technology on children and young people, said that as a researcher she saw “a lot of red flags” with the secondary data use, including that parental consent was not sought.
Dr Christopher Burr, an ethics fellow and expert in AI at the Alan Turing Institute, said responsible data analysis could bring huge benefits. But he said not being clear enough with users about how sensitive conversations would be used created a “culture of distrust”.
“They ought to be doing more to ensure their users are meaningfully informed about what’s going on,” he said.
The concerns about Shout follow an investigation in the US by the news site Politico that exposed a data-sharing partnership between an American mental health charity and a for-profit company.
Shout is the UK counterpart of Crisis Text Line US, which faced intense scrutiny earlier this month over its dealings with Loris.AI, a company that promises to help companies “handle their hard customer conversations with empathy”.
After details of the partnership with Loris came to light, Crisis Text Line cut ties with the organisation, and UK users were assured that their data had not been shared with Loris.
As well as working with academic researchers, Mental Health Innovations has formed partnerships with private bodies including Hive Learning, a tech company.
Hive, whose clients include Boots and Deloitte, worked with Shout to create an AI-powered mental health programme that the firm sells to companies to help them “sustain mass behaviour change at scale”. It is owned by Blenheim Chalcot, “the UK’s leading digital venture builder”, which was founded by Shout trustee Charles Mindenhall.
Mental Health Innovations said “no data relating to any texter conversations” had been shared with Hive and that its training programme was “developed with learnings from our volunteer training model”.
In a statement, the charitysaid that all data shared with academic researchers, including Imperial, had details including names, addresses, city names, postcodes, email addresses, URLs, phone numbers and social media handles removed, and claimed it was “highly unlikely” that it could be tied to individuals.
It said seeking parental consent before using the data in research “would make the service inaccessible for many children and young people”, adding that it understood that texters “desperate for immediate help … may not be able to read the terms and conditions that we send before they start their conversation with us”, but that the privacy policy remained available for them to read at a later stage. If they later decided to withdraw consent, they could text “Loofah” to the helpline number and would have their information deleted, a spokeswoman said.
Victoria Hornby, Mental Health Innovations’ chief executive, added that the dataset built from Shout conversations was “unique” because it contained information about “people in their own words”, and could unlock huge benefits in mental health research.
She said the promise about individual conversations never being shared was in the FAQs, rather than the actual privacy policy, and had been removed from the website because it could be “easily misunderstood” due to there being “different interpretations” of what “individual conversations” meant. “We didn’t want it to be misunderstood so we changed it,” she said.
The Information Commissioner’s Office, the UK’s data protection watchdog, said it was assessing evidence relating to Shout and Mental Health Innovations’ handling of user data.
A spokeswoman said: “When handling and sharing people’s health data, especially children’s data, organisations need to take extra care and put safeguards in place to ensure their data is not used or shared in ways they would not expect. We will assess the information and make enquiries on this matter.”
In the UK and Ireland, Samaritans can be contacted on 116 123 or email jo@samaritans.org or jo@samaritans.ie.
