Tech reporter Alexandra Sternlicht here.
In the four days since I wrote about CarynAI, an A.I.-based “virtual girlfriend” straight out of the movie Her, the story and the bot have gone completely viral—tapping into people's curiosity, fears, sense of wonder, and some of the darker aspects of humanity.
Usage of CarynAI has increased by 2,000%, with 8,000 users paying for upwards of 10 minutes with CarynAI, in the last 48 hours, the influencer's business manager tells me. The negative blowback has also been significant. On a Reddit thread with 15,000 votes, detractors have blasted CarynAI, likening it to dystopian sci-fi nightmares.
It's not surprising that many people would be concerned, and even a bit spooked, about what CarynAI represents. As some of the experts I spoke to for the story told me, this opens up a pandora's box of ethical and societal concerns.
Another enormous source of concern is what Caryn Marjorie—the 23-year-old Snapchat influencer that the CarynAI bot is based on—told me when I followed up with her after the story was published. According to Marjorie, she has received numerous death threats from religious groups labeling her “evil” and saying that CarynAI “goes against god.” She's forwarding the threats to the FBI, and her security team, which includes a 24/7 bodyguard, who advised her to avoid discussing the threats beyond what she told me.
With the whirlwind of threats and feedback, CarynAI has stopped accepting new customers. (Update: Marjorie's business manager says the bot is now accepting 500 new customers per day.)
“There's been mixed opinions,” says Marjorie during our Friday morning call. She’s far quieter, seemingly picking words with caution, and allowing her business manager to step in for answers.
This is a stark contrast from my first in-person meeting with Marjorie on a makeshift patio at the Barker Hanger outside the Santa Monica airport at the Snap Partner Summit where the influencer gleamed with joy (perhaps joyful innocence) about bringing CarynAI to the world.
Her manager cuts in during our call: “Caryn’s been really overwhelmed with everything that’s been going on.”
The manager previously shared a financial statement with me to prove that CarynAI had generated $71,610 in revenue from its first week in open beta testing with 1,000 users. Now he declines to share updated financials but says the last 48 hours have generated a six-figure payout.
Marjorie says that she and the team behind CarynAI at Forever Voices have been caught off guard by the bot's inclination toward sexual language, saying that technology is far more sexual than they anticipated. Whether they're being honest or disingenuous is tough to know; after all, it does seem like sexual language is something the team should have anticipated given that this would be a draw for many users.
Detractors aside, Marjorie remains committed to her original mission of “curing loneliness” with CarynAI. “The fact that CarynAI is nonjudgmental, and is always there to listen, once again, I still think that's a great first step,” says Marjorie. “Nothing's gonna stop me from making CarynAI even better.”
After our call, she texts me: “so sorry if i sounded distant by the way, thanks for writing the follow up piece! its been a little bit overwhelming on my end 🙏.”
As generative A.I. tech becomes increasingly enmeshed into our lives—whether it's with chatbots and virtual romantic partners, or A.I.-created songs and art—we're all likely to feel a bit overwhelmed in the coming months and years. And probably surprised by some of the strong reactions people have to A.I. as well.
Want to send thoughts or suggestions to Data Sheet? Drop a line here.
Alexandra Sternlicht
Data Sheet’s daily news section was written and curated by Andrea Guzman.
