An AI-generated Seinfeld show has been banned from the streaming platform Twitch for at least 14 days after a transphobic and homophobic standup bit aired during the show.
Nothing, Forever, a 24/7 version and AI-generated parody of the popular Seinfeld sitcom, had been available on Twitch since mid-December.
The show follows four characters – Larry, Fred, Yvonne and Kakler – who spend the never-ending stream discussing their days and other minor details, similar to Seinfeld and its four main characters: Jerry, George, Elaine and Kramer.
Mimicking Seinfeld, the AI stream opens up with its character Larry performing a standup routine at the show’s beginning.
But during a stream on Sunday night, Larry made a series of homophobic and transphobic remarks during a standup bit, according to a clip on LiveStreamFails.com.
“I’m thinking about doing a bit about how being transgender is actually a mental illness,” Larry said on the clip.
“Or how all liberals are secretly gay and want to impose their will on everyone. Or something about how transgender people are ruining the fabric of society.”
The animated stream has been suspended for at least two weeks in light of those remarks, one of the show’s creators, Xander, said on the online chatting platform Discord, according to Vice News.
“Hey everybody. Here’s the latest: we received a 14-day suspension due to what Larry Feinberg said tonight during a club bit,” Xander reportedly said.
“We’ve appealed the ban, and we’ll let you know as we know more on what Twitch decides. Regardless of the outcome of the appeal, we’ll be back and will spend the time working to ensure to the best of our abilities that nothing like that happens again.”
Staff commented on a Discord thread about the fiasco, blaming a switch in the AI model for the characters’ “errant behaviors”, reported Indie Wire.
“Earlier tonight, we started having an outage using OpenAI’s GPT-3 Davinci model, which caused the show to exhibit errant behaviors (you may have seen empty rooms cycling through),” a staff member using the handle tinylobsta wrote.
“OpenAI has a less sophisticated model, Curie, that was the predecessor to Davinci. When Davinci started failing, we switched over to Curie to try to keep the show running without any downtime. The switch to Curie was what resulted in the inappropriate text being generated.”
The dialogue used in Nothing, Forever is generated from the text generator OpenAI’s GPT-3, with the company Mismatch Media responsible for the tech behind the speech.
In an email to the Guardian, Skyler Hartle of Mismatch Media said that the text generated “was a mistake”.
“We are super embarrassed, and that the generative content created in no way reflects the values or opinions of our staff,” Hartle said.
Hartle added that the issue was caused due to OpenAI’s GPT-3 Davinci model “not correctly leveraging their content moderation systems”.
“Before going live again, we are going to implement the correct content moderation tools OpenAI provides and also investigate using a secondary content moderation system as a redundancy,” Hartle said. “We very much regret this happened and hope to be back on the air soon, with all the appropriate safeguards in place.”
Twitch could not immediately be reached for comment on the show’s suspension.
