AI watermarking is an unreliable technology with ‘significant limitations’, OpenAI said.

OpenAI is rolling out text watermarking in the EU in its latest attempt to comply with the region’s AI laws after the bloc said it will regulate ChatGPT under its strictest rules for online platforms.

The company said that it will add an invisible watermark over the coming weeks to text generated from ChatGPT and Codex in the region. API customers globally will also be allowed to opt in to text watermarking for select models, while select researchers and experts will get access to test out the watermark detector.

The new feature comes after transparency rules under the EU AI Act came into force this August which require certain AI systems to tell users when they are interacting with AI and how the content they are consuming is generated or altered by it.

AI watermarking technology is not entirely reliable, researchers, AI companies and lawmakers all agree. But concerns around the impact of manipulated content on society means regulators are pushing for detection technology, regardless of its accuracy at this moment.

OpenAI’s text watermarking technology, called TextGrain, adds an invisible statistical signal to the model’s word choices, the company said. The detector looks for that signal to assess whether a passage contains a watermark.

Though shorter passages and AI-generated output edited by humans are difficult to detect, the company clarified. According to its testing, detection rates plummet from close to 100pc when generated texts are unedited, to less than 20pc when only 25pc of the words are changed.

“Text watermarking and detection remain early technologies with significant limitations… [and] our phased approach reflects both the EU AI Act requirements as well as the technology’s limitations,” OpenAI said.

The ChatGPT-maker is one of 200 signatories of the Act’s Code of Practice on Transparency of AI-Generated Content – a voluntary tool aimed at helping businesses comply with the sweeping laws. Anthropic, France’s Mistral, Meta and Microsoft are also signatories.

OpenAI already has a number of verification mechanisms for other types of content. Supported image and audio outputs are labelled with C2PA and SynthID watermarks.

Anthropic, in August, pledged to start watermarking content in the EU. According to the company, text generated using Claude models launched in the bloc after 2 August now carry embedded watermarks, and generated files will include the C2PA credentials wherever supported.

Don’t miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic’s digest of need-to-know sci-tech news.