Hello and welcome to Eye on AI. In this week’s edition: The difficulty of labeling AI-generated content; a bunch of new reasoning models are nipping at OpenAI’s heels; Google DeepMind uses AI to correct quantum computing errors; the sun sets on human translators.
With the U.S. presidential election behind us, it seems like we may have dodged a bullet on AI-generated misinformation. While there were plenty of AI-generated memes bouncing around the internet, and evidence that AI was used to create some misleading social media posts—including by foreign governments attempting to influence voters—there is so far little indication AI-generated content played a significant role in the election’s outcome.
That is mostly good news. It means we have a bit more time to try to put in place measures that would make it easier for fact-checkers, the news media, and average media consumers to determine if a piece of content is AI-generated. The bad news, however, is that we may get complacent. AI’s apparent lack of impact on the election may remove any sense of urgency to putting the right content authenticity standards in place.
C2PA is winning out—but it's far from perfect
While there have been a lot of suggestions for authenticating content and recording its provenance information, the industry seems to be coalescing, for better or worse, around C2PA’s content credentials. C2PA is the Coalition for Content Provenance and Authenticity, a group of leading media organizations and technology vendors who are jointly promulgating a standard for cryptographically signed metadata. The metadata includes information on how the content was created, including whether AI was used to generate or edit it. C2PA is often erroneously conflated with “digital watermarking” of AI outputs. The metadata can be used by platforms distributing content to inform content labeling or watermarking decisions, but is not itself a visible watermark—nor is it an indelible digital signature that can’t be stripped from the original file.
But the standard still has a lot of potential issues, some of which were highlighted by a recent case study looking at how Microsoft-owned LinkedIn had been wrestling with content labeling. The case study was published by the Partnership on AI (PAI) earlier this month and was based on information LinkedIn itself provided in response to an extensive questionnaire. (PAI is another nonprofit coalition founded by some of the leading technology companies and AI labs, along with academic researchers and civil society groups, that works on creating standards around responsible AI.)
LinkedIn applies a visible “CR” label in the upper lefthand corner of any content uploaded to its platform that has C2PA content credentials. A user can then click on this label to reveal a summary of some of the C2PA metadata: the tool used to create the content, such as the camera model, or the AI software that generated the image or video; the name of the individual or entity that signed the content credentials; and the date and time stamp of when the content credential was signed. LinkedIn will also tell the user if AI was used to generate all or part of an image or video.
Most people aren't applying C2PA credentials to their stuff
One problem is that the system is entirely dependent on whoever creates the content applying C2PA credentials. Only a few cameras or smart phones currently apply these by default. Some AI image generation software—such as OpenAI’s DALLE-3 or Adobe’s generative AI tools—do apply the C2PA credentials automatically, although users can opt out of these in some Adobe products. But for video, C2PA remains largely an opt in system.
I was surprised to discover, for instance, that Synthesia, which produces highly realistic AI avatars, is not currently labeling its videos with C2PA by default, even though Synthesia is a PAI member, has done a C2PA pilot, and its spokesperson says the company is generally supportive of the standard. “In the future, we are moving to a world where if something doesn’t have content credentials, by default you shouldn’t trust it,” Alexandru Voica, Synthesia’s head of corporate affairs and policy, told me.
Voica is a prolific LinkedIn user himself, often posting videos to the professional networking site featuring his Synthesia-generated AI avatar. And yet, none of Voica’s videos had the “CR” label or carried C2PA certificates.
C2PA is currently “computationally expensive,” Voica said. In some cases, C2PA metadata can significantly increase a file’s size, meaning Synthesia would need to spend more money to process and store those files. He also said that, so far, there’s been little customer demand for Synthesia to implement C2PA by default, and that the company has run into an issue where the video encoders many social media platforms use strip the C2PA credentials from the videos uploaded to the site. (This was a problem with YouTube until recently, for instance; now the company, which joined C2PA earlier this year, supports content credentials and applies a “made with a camera” label to content that carries C2PA metadata indicating it was not AI manipulated.)
LinkedIn—in its response to PAI’s questions—cited challenges with the labeling standard including a lack of widespread C2PA adoption and user confusion about the meaning of the “CR” symbol. It also noted Microsoft’s research about how "very subtle changes in language (e.g., ‘certified’ vs. ‘verified’ vs. ‘signed by’) can significantly impact the consumer’s understanding of this disclosure mechanism.” The company also highlighted some well-documented security vulnerabilities with C2PA credentials, including the ability of a content creator to provide fraudulent metadata before applying a valid cryptographic signature, or someone screenshotting the content credentials information LinkedIn displays, editing this information with photo editing software, and then reposting the edited image to other social media.
More guidance on how to apply the standard is needed
In a statement to Fortune, LinkedIn said “we continue to test and learn as we adopt the C2PA standard to help our members stay more informed about the content they see on LinkedIn.” The company said it is “continuing to refine” its approach to C2PA: “We’ve embraced this because we believe transparency is important, particularly as [AI] technology grows in popularity."
Despite all these issues, Claire Leibowicz, the head of the AI and media integrity program at PAI, commended Microsoft and LinkedIn for answering PAI’s questions candidly and being willing to share some of the internal debates they’d had about how to apply content labels.
She noted that many content creators might have good reason to be reluctant to use C2PA, since an earlier PAI case study on Meta’s content labels found that users often shunned content Meta had branded with an “AI-generated” tag, even if that content had only been edited with AI software or was something like a cartoon, in which the use of AI had little bearing on the informational value of the content.
As with nutrition labels on food, Leibowicz said there was room for debate about exactly what information from C2PA metadata should be shown to the average social media user. She also said that greater C2PA adoption, improved industry-consensus around content labeling, and ultimately some government action would help—and she noted that the U.S. National Institute of Standards and Technology was currently working on a recommended approach. Voica had told me that in Europe, while the EU AI Act doesn’t mandate content labeling, it does say that all AI-generated content must be “machine readable,” which ought to help bolster adoption of C2PA.
So it seems C2PA is likely to be here to stay, despite the protests of security experts who would prefer a system that is less dependent on trust. Let’s just hope the standard is more widely adopted—and that C2PA works to fix its known security vulnerabilities—before the next the election cycle rolls around. With that, here’s more AI news.
Programming note: Eye on AI will be off on Thursday for the Thanksgiving holiday in the U.S. It'll be back in your inbox next Tuesday.
Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn
**Before we get the news: There's still time to apply to join me in San Francisco for the Fortune Brainstorm AI conference! If you want to learn more about what's next in AI and how your company can derive ROI from the technology, Fortune Brainstorm AI is the place to do it. We'll hear about the future of Amazon Alexa from Rohit Prasad, the company's senior vice president and head scientist, artificial general intelligence; we'll learn about the future of generative AI search at Google from Liz Reid, Google's vice president, search; and about the shape of AI to come from Christopher Young, Microsoft's executive vice president of business development, strategy, and ventures; and we'll hear from former San Francisco 49er Colin Kaepernick about his company Lumi and AI's impact on the creator economy. The conference is Dec. 9-10 at the St. Regis Hotel in San Francisco. You can view the agenda and apply to attend here. (And remember, if you write the code KAHN20 in the "Additional comments" section of the registration page, you'll get 20% off the ticket price—a nice reward for being a loyal Eye on AI reader!)
