Labeling AI-generated content is not as easy as it seems

C2PA is winning out—but it’s far from perfect

While there have been a lot of suggestions for authenticating content and recording its provenance information, the industry seems to be coalescing, for better or worse, around C2PA’s content credentials. C2PA is the Coalition for Content Provenance and Authenticity, a group of leading media organizations and technology vendors who are jointly promulgating a standard for cryptographically signed metadata. The metadata includes information on how the content was created, including whether AI was used to generate or edit it. C2PA is often erroneously conflated with “digital watermarking” of AI outputs. The metadata can be used by platforms distributing content to inform content labelling or watermarking decisions, but is not itself a visible watermark—nor is it an indelible digital signature that can’t be stripped from the original file.

But the standard still has a lot of potential issues, some of which were highlighted by a recent case study looking at how Microsoft-owned LinkedIn had been wrestling with content labelling. The case study was published by the Partnership on AI (PAI) earlier this month and was based on information LinkedIn itself provided in response to an extensive questionnaire. (PAI is another nonprofit coalition founded by some of the leading technology companies and AI labs, along with academic researchers and civil society groups, that works on creating standards around responsible AI.)

LinkedIn applies a visible “CR” label in the upper lefthand corner of any content uploaded to its platform that has C2PA content credentials. A user can then click on this label to reveal a summary of some of the C2PA metadata: the tool used to create the content, such as the camera model, or the AI software that generated the image or video; the name of the individual or entity that signed the content credentials; and the date and time stamp of when the content credential was signed. LinkedIn will also tell the user if AI was used to generate all or part of an image or video.

Most people aren’t applying C2PA credentials to their stuff

One problem is that currently the system is entirely dependent on whoever creates the content applying C2PA credentials. Only a few cameras or smart phones currently apply these by default. Some AI image generation software—such as OpenAI’s DALLE-3 or Adobe’s generative AI tools—do apply the C2PA credentials automatically, although users can opt out of these in some Adobe products. But for video, C2PA remains largely an opt in system.

I was surprised to discover, for instance, that Synthesia, which produces highly realistic AI avatars, is not currently labelling its videos with C2PA by default, even though Synthesia is a PAI member, has done a C2PA pilot, and its spokesperson says the company is generally supportive of the standard. “In the future, we are moving to a world where if something doesn’t have content credentials, by default you shouldn’t trust it,” Alexandru Voica, Synthesia’s head of corporate affairs and policy, told me.

Voica is a prolific LinkedIn user himself, often posting videos to the professional networking site featuring his Synthesia-generated AI avatar. And yet, none of Voica’s videos had the “CR” label or carried C2PA certificates.

C2PA is currently “computationally expensive,” Voica said. In some cases, C2PA metadata can significantly increase a file’s size, meaning Synthesia would need to spend more money to process and store those files. He also said that, so far, there’s been little customer demand for Synthesia to implement C2PA by default, and that the company has run into an issue where the video encoders many social media platforms use strip the C2PA credentials from the videos uploaded to the site. (This was a problem with YouTube until recently, for instance; now the company, which joined C2PA earlier this year, supports content credentials and applies a “made with a camera” label to content that carries C2PA metadata indicating it was not AI manipulated.)

LinkedIn—in its response to PAI’s questions—cited challenges with the labelling standard including a lack of widespread C2PA adoption and user confusion about the meaning of the “CR” symbol. It also noted Microsoft’s research about how “very subtle changes in language (e.g., ‘certified’ vs. ‘verified’ vs. ‘signed by’) can significantly impact the consumer’s understanding of this disclosure mechanism.” The company also highlighted some well-documented security vulnerabilities with C2PA credentials, including the ability of a content creator to provide fraudulent metadata before applying a valid cryptographic signature, or someone screenshotting the content credentials information LinkedIn displays, editing this information with photo editing software, and then reposting the edited image to other social media.

More guidance on how to apply the standard is needed

In a statement to Fortune, LinkedIn said “we continue to test and learn as we adopt the C2PA standard to help our members stay more informed about the content they see on LinkedIn.” The company said it is “continuing to refine” its approach to C2PA: “We’ve embraced this because we believe transparency is important, particularly as [AI] technology grows in popularity.”

Despite all these issues, Claire Leibowicz, the head of the AI and media integrity program at PAI, commended Microsoft and LinkedIn for answering PAI’s questions candidly and being willing to share some of the internal debates they’d had about how to apply content labels.

She noted that many content creators might have good reason to be reluctant to use C2PA, since an earlier PAI case study on Meta’s content labels found that users often shunned content Meta had branded with an “AI-generated” tag, even if that content had only been edited with AI software or was something like a cartoon, in which the use of AI had little bearing on the informational value of the content.

As with nutrition labels on food, Leibowicz said there was room for debate about exactly what information from C2PA metadata should be shown to the average social media user. She also said that greater C2PA adoption, improved industry-consensus around content labelling, and ultimately some government action would help—and she noted that the U.S. National Institute of Standards and Technology was currently working on a recommended approach. Voica had told me that in Europe, while the EU AI Act doesn’t mandate content labelling, it does say that all AI-generated content must be “machine readable,” which ought to help bolster adoption of C2PA.

So it seems C2PA is likely to be here to stay, despite the protests of security experts who would prefer a system that less dependent on trust. Let’s just hope the standard is more widely adopted—and that C2PA works to fix its known security vulnerabilities—before the next the election cycle rolls around. With that, here’s more AI news.

Programming note: Eye on AI will be off on Thursday for the Thanksgiving holiday in the U.S. It’ll be back in your inbox next Tuesday.

Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn

**Before we get the news: There’s still time to apply to join me in San Francisco for the Fortune Brainstorm AI conference! If you want to learn more about what’s next in AI and how your company can derive ROI from the technology, Fortune Brainstorm AI is the place to do it. We’ll hear about the future of Amazon Alexa from Rohit Prasad, the company’s senior vice president and head scientist, artificial general intelligence; we’ll learn about the future of generative AI search at Google from Liz Reid, Google’s vice president, search; and about the shape of AI to come from Christopher Young, Microsoft’s executive vice president of business development, strategy, and ventures; and we’ll hear from former San Francisco 49er Colin Kaepernick about his company Lumi and AI’s impact on the creator economy. The conference is Dec. 9-10 at the St. Regis Hotel in San Francisco. You can view the agenda and apply to attend here. (And remember, if you write the code KAHN20 in the “Additional comments” section of the registration page, you’ll get 20% off the ticket price—a nice reward for being a loyal Eye on AI reader!)

AI IN THE NEWS

U.S. Justice Department seeks to unwind Google’s partnership with Anthropic. That’s one of the remedies the department’s lawyers are seeking from a federal judge who has found Google maintains an illegal monopoly over online search, Bloomberg reported. The proposal would bar Google from acquiring, investing in, or collaborating with companies controlling information search, including AI query products, and requires divestment of Chrome. Google criticized the proposal, arguing it would hinder AI investments and harm America’s technological competitiveness.

Coca-Cola’s AI-generated Christmas ads spark a backlash. The company used AI to help create its Christmas ad campaign—which contains nostalgic elements such as Santa Claus and cherry-red Coca-Cola trucks driving through snow-blanketed towns, and which pay homage to an ad campaign the beverage giant ran in the mid-1990s. But some say the ads feel unnatural, while others accuse the company of undermining the value of human artists and animators, the New York Times reported. The company defended the ads saying they were simply the latest in a long tradition of Coke “capturing the magic of the holidays in content, film, events and retail activations.”

More companies debut AI reasoning models, including open-source versions. A clutch of OpenAI competitors launched AI models that they claim are competitive, or even better performing, than OpenAI’s o1-preview model, which was designed to excel at tasks that require reasoning, including mathematics and coding, tech publication The Information reported. The companies include Chinese internet giant Alibaba, which launched an open-source reasoning model, but also little-known startup Fireworks AI and a Chinese quant trading firm called High-Flyer Capital. It turns out it is much easier to develop and train a reasoning model than a traditional large language model. The result is that OpenAI, which had hoped its o1 model would give it a substantial lead on competitors, has more rivals nipping at its heels than anticipated just three months after it debuted o1-preview.

Trump weighs appointing an AI czar. That’s according to a story in Axios that says billionaire Elon Musk and entrepreneur and former Republican party presidential contender Vivek Ramaswamy, who are jointly heading up the new Department of Government Efficiency (DOGE), will have a significant voice in shaping the role and deciding who gets chosen for it, although neither was expected to take the position themselves. Axios also reported that Trump was not yet decided on whether to create the role, which could be combined with a cryptocurrency czar, to create an overall emerging-technology role within the White House. 

EYE ON AI RESEARCH

Google DeepMind uses AI to improve error correction in a quantum computer. Google has developed AlphaQubit, an AI model that can correct errors in the calculations of a quantum computer with a high degree of accuracy. Quantum computers have the potential to solve many kinds of complex problems much faster than conventional computers, but today’s quantum circuits are highly prone to calculation errors due to electromagnetic interference, heat, and even vibrations. Google DeepMind worked with experts from Google’s Quantum AI team to develop the AI model.

While very good at finding and correcting errors, the AI model is not fast enough to correct errors in real-time, as a quantum computer is running a task, which is what will really be needed to make quantum computers more effective for most real-world applications. Real-time error correction is especially important for quantum computers built using qubits made from superconducting materials, as these circuits can only remain in a stable quantum state for brief fractions of a second.

Still, AlphaQubit is a step towards eventually developing more effective, and potentially real-time, error correction. You can read Google DeepMind’s blog post on AlphaQubit here.

FORTUNE ON AI

Most Gen Zers are terrified of AI taking their jobs. Their bosses consider themselves immune —by Chloe Berger

Elon Musk’s lawsuit could be the least of OpenAI’s problems—shedding its nonprofit status will cost a fortune —by Christiaan Hetzner

Sam Altman has an idea to get AI to ‘love humanity,’ use it to poll billions of people about their value systems —by Paolo Confino

The CEO of Anthropic blasts VC Marc Andreessen’s argument that AI shouldn’t be regulated because it’s ‘just math’ —by Kali Hays

AI CALENDAR

Dec. 2-6: AWS re:Invent, Las Vegas

Dec. 8-12: Neural Information Processing Systems (Neurips) 2024, Vancouver, British Columbia

Dec. 9-10: Fortune Brainstorm AI, San Francisco (register here)

Dec. 10-15: NeurlPS, Vancouver

Jan. 7-10: CES, Las Vegas

Jan. 20-25: World Economic Forum. Davos, Switzerland

BRAIN FOOD

AI translation is fast eliminating the need for human translators for business

That was the revealing takeaway from my conversation at Web Summit earlier this month with Unbabel’s cofounder and CEO Vasco Pedro and his cofounder and CTO, João Graça. Unbabel began life as a marketplace app, pairing companies that needed translation, with freelance human translators—as well as offering machine translation options that were superior to what Google Translate could provide. (It also developed a quality model that can check the quality of a particular translation.) But, in June, Unbabel developed its own large language model, called TowerLLM, that beat almost every LLM on the market in its translation between English and Spanish, French, German, Portuguese, Italian, and Korean. The model was particularly good at what’s called “transreation”—not word-for-word, literal translation, but understanding when a particular colloquialism is needed or when cultural nuance requires deviation from the original text to convey the correct connotations. TowerLLM was soon powering 40% of the translation jobs contracted over Unbabel’s platform, Graça said.

At Web Summit, Unbabel announced a new standalone product called Widn.AI that is powered by its TowerLLM and offers customers translations across more than 20 languages. For most business use cases, including technical domains such as law, finance, or medicine, Unbabel believes its Widn product can now offer translations that are every bit as good—if not better—than what an expert human translator would produce, Graça tells me.

He says human translators will increasingly need to migrate to other work, while some will still be needed  to supervise and check the output of AI models such as Widn in contexts where there is a legal requirement that a human certify the accuracy of a translation—such as court submissions. Humans will still be needed to check the quality of the data being fed AI models too, Graça said, although even some of this work can now be automated by AI models. There may still be some role for human translators in literature and poetry, he allows—although here again, LLMs are increasingly capable (for instance, making sure a poem rhymes in the translated language without deviating too far from the poem’s original meaning, which is a daunting translation challenge).

I, for one, think human translators aren’t completely going to disappear. But it is hard to argue that we will need as many of them. And this is a trend we might see play out in other fields too. While I’ve generally been optimistic that AI will, like every other technology before it, ultimately create more jobs than it destroys—this is not the case in every area. And translation may be one of the first casualties. What do you think?