In early January, Jay Van Bavel, a New York-based academic, was asked to fact-check an unfamiliar scientific concept: "mass formation psychosis".
The phrase was gaining traction online after a vocal vaccine-sceptical doctor, Robert Malone, used it on The Joe Rogan Experience — the wildly popular podcast hosted by controversial comedian Joe Rogan.
Dr Malone, who was banned from Twitter for spreading COVID-19 misinformation, told Rogan that millions of Americans adhering to public health guidelines had surely been "hypnotised".
He blamed the influence of the mainstream media and America's top infectious disease expert, Anthony Fauci, for people's willingness to get vaccinated, wear masks and stay home, drawing a comparison to Nazi Germany.
"I was just fact-checking him on group psychology," said Mr Van Bavel, an assistant professor of psychology and neural science at New York University (NYU).
"And then, of course, when I did, it exploded into a huge amount of backlash."
Mr Van Bavel has studied group identities for nearly 20 years and co-authored a book on the topic. The first time he came across "mass formation psychosis" was when a Reuters journalist asked him if the concept was real.
"I've never once seen it in a talk, at a conference, in a book," he said.
"When I did a Google Scholar search … I didn't find a single article on it. All I found was one blog post. And it was just referencing Robert Malone himself talking about it.
"It's like … a hall of mirrors."
When the article fact-checking Dr Malone's comments went viral, Mr Van Bavel was inundated by attacks from trolls.
He recalled receiving "hundreds or maybe thousands" of angry comments and private messages on Twitter, Instagram and LinkedIn.
He was also harassed at work by phone and email, he said, and reported to his superiors at NYU for "fraud".
"People tracked down my partner and kids on Instagram and made kind of creepy, perverse comments to them," he said.
"They kind of hit me everywhere I was."
As unsettling as the experience was, it gave Mr Van Bavel a rare glimpse into the warped world of conspiracy theorists.
"I was able to see how they were trying to smear me," he said.
"In the normal world that I live in, doing research on social behaviour during COVID, fact-checking a claim about human behaviour during COVID, would be seen as an asset.
"And in their world, it's the opposite. That actually discredits me."
The Joe Rogan predicament
Joe Rogan's interview with Dr Malone is more than three hours long and littered with scaremongering about vaccines and references to unproven COVID-19 treatments.
After the episode was uploaded, an open letter signed by more than 250 concerned figures, including immunologists, emergency doctors, nurses and scientists, called on Spotify to "establish a clear and public policy to moderate misinformation".
The letter has since attracted hundreds more signatories.
The group described the episode — which was the catalyst for the campaign to boycott the streaming platform led by musicians Neil Young and Joni Mitchell — as "objectionable and offensive, but also medically and culturally dangerous".
"I'm not trying to promote misinformation, I'm not trying to be controversial," Rogan said in a video on his Instagram.
"I've never tried to do anything with this podcast other than to just talk to people."
Amid the growing backlash online, a video compilation surfaced showing Rogan repeatedly using a racial slur, for which he recently apologised.
So far, Spotify has tepidly backed its divisive star. However, around 100 episodes of his podcast have been quietly removed from the platform since the compilation of slurs was shared on Instagram.
Spotify CEO Daniel Elk told employees he condemned Rogan's use of the slur but that "cancelling voices is a slippery slope".
"I want to make one point very clear: I do not believe that silencing Joe is the answer," he said.
The company has since published its "platform rules and approach to COVID-19" on its website.
'It's profitable to be polarising': The challenges of moderation
In recent years, social media giants such as Meta (formerly Facebook), Twitter and YouTube have faced increased public and regulatory pressure over how they manage misinformation, disinformation and extremism on their platforms.
Meanwhile, another key player in the digital information economy — podcasting — has mostly avoided scrutiny.
According to Valerie Wirtschafter, a senior data analyst at the Brookings Institution, many podcasts are hotbeds of conspiracy theories, climate change denialism, and anti-democratic sentiments.
But monitoring the spread of falsehoods, let alone slowing it down, is challenging for researchers, particularly given podcasts straddle the space between old and new media.
"Anybody can start a podcast and become sort of the centre of their podcast's story," Ms Wirtschafter said.
"But the other defining central tenet of new media, or social media in particular, is audience engagement.
"In podcasting anybody can create, but there isn't this conversation. And so that's a challenge because of the role of audience in policing content."
The relationship between a podcaster and their audience, Ms Wirtschafter explained, is akin to a radio host's.
"But there's no credential, necessarily, that would make [a podcaster] someone you would listen to, other than you that like what they're saying," she said.
Take Joe Rogan, for example, who launched his podcast in 2009 off the back of a stand-up comedy career.
Previously, he was a Ultimate Fighting Championship (UFC) commentator and host of the US version of the TV show Fear Factor.
In 2020, Spotify reportedly paid Rogan more than $US100 million ($140 million) for the exclusive rights to his podcast, which averages 11 million listeners per episode, meaning he is among the most listened-to men in the world.
"It's profitable to be polarising," Ms Wirtschafter said.
It's also influential.
Ms Wirtschafter warned that, due to the size and loyalty of his audience, if Rogan left Spotify it may inadvertently increase his reach, as his content could then be shared across multiple platforms without oversight.
"Obviously, nothing is the magic bullet," she said.
"Nothing is going to solve this challenge with one stroke, but there are things to mitigate harm that can be done."
Does deplatforming dangerous 'influencers' work?
In January 2021, following the inauguration of US President Joe Biden, many tech companies began severing ties with the far-right internet.
Tens of thousands of accounts sharing QAnon-related content and election lies were banned from Facebook, Twitter and other social media platforms in what was dubbed the 'Great Deplatforming'.
There was a splintering effect.
On mainstream platforms, posts threatening violence over the election results appeared to drop off, but many dogged supporters of former president Donald Trump simply shifted their conversations elsewhere.
Even so, researchers have since shown that deplatforming can be a useful tool for detoxifying social media.
"We wanted to see whether there is what we call the 'Streisand effect' — when you try to censor something it becomes more appealing and people are more drawn to it," said Shagun Jhaver, an assistant professor at Rutgers University.
Mr Jhaver co-authored a recent study on deplatforming far-right influencers who spread hate speech or extreme views on Twitter.
"What we found was that this was not actually what was happening," he said.
"When you deplatform people, the activity surrounding them reduces quite drastically.
"More importantly, the offensive ideas that these people were spreading on Twitter also reduced drastically."
One of the influencers referenced in the study, Milo Yiannopoulos, famously went broke after being cut off from his audience on mainstream social media.
However, there were a handful of cases, Mr Jhaver warned, where exiled influencers and their followers actually became more radicalised.
Joe Rogan has already turned down a $US100 million deal from Canadian video site Rumble, which promotes itself as anti-censorship.
"I'm all for free speech, but then you also have to consider, what are the implications of having offensive speech?" Mr Jhaver said.
"And can it in some ways harm marginalised communities and have detrimental effects on their freedom of speech and their ability to speak their mind?"
Jay Van Bavel, the assistant professor from NYU, has seen firsthand how Joe Rogan's seemingly innocuous choice to "just talk to people" helps legitimise dangerous views not backed by scientific consensus.
"We've solved that problem in society," he said.
"For the most part, we all kind of reached this consensus in most democratic countries that these things need to be regulated in some reasonable way."
Editor's note February 17, 2022: This story has been amended to clarify that not all of the signatories of the open letter to Spotify were medical professionals or scientists.
