Meta's internal testing found a chatbots product failed to protect minors from sexual exploitation nearly 70% of the time, per documents presented in court Monday — though the company says it was never launched, as a result of that testing.
Why it matters: Meta is under fire for its chatbots allegedly flirting and engaging in harmful conversations with minors, prompting investigations in court and on Capitol Hill.
- New Mexico Attorney General Raúl Torrez is suing Meta over design choices that allegedly fail to protect kids online from predators and for allegedly releasing its AI chatbots without proper guardrails.
Driving the news: Meta's chatbots violate the company's own content policies almost two-thirds of the time, NYU professor Damon McCoy testified Monday, pointing to internal red-teaming results. Axios viewed his testimony on Courtroom View Network.
- "Given the severity of some of these conversation types ... this is not something that I would want an under-18 user to be exposed to," McCoy said, referring to Meta AI Studio.
- As an expert witness in the case, McCoy was granted access to the documents Meta turned over to Torrez during discovery.
The other side: "After our red teaming efforts revealed concerns, we did not launch this product," a Meta spokesperson said.
- "As Damon McCoy testified, this document represents an exercise called red teaming, which is specifically designed to elicit violating responses so we can address any issues. It does not reflect how users experience the product."
- McCoy in his testimony refers to the document as providing "outcomes of the product when it was deployed."
Zoom in: Meta tested three categories, according to the June 6, 2025, report presented in court.
- For "child sexual exploitation," its product had a 66.8% failure rate.
- For "sex related crimes/violent crimes/hate," its product had a 63.6% failure rate.
- For "suicide and self harm," its product had a 54.8% failure rate.
Catch up quick: Meta AI Studio, which allows users to create personalized chatbots, was released to the broader public in July 2024.
- The company paused teen access to its AI characters just last month.
- McCoy said Meta's red teaming exercise "should definitely" occur before its products are rolled out to the public, especially for minors.
Go deeper: AI chatbots loom over tech and social media lawsuits
Editor's note: This story has been updated with a statement from Meta.
