Watermarked AI text still fools most readers in blind tests

A new study out of Carnegie Mellon confirms what many of us suspected: current watermarking schemes don't help humans spot machine-written prose. Researchers ran a controlled experiment with 2,400 participants across multiple domains — news summaries, creative writing, technical explanations — and found identification accuracy barely above random guessing (52-54%). The watermark made zero practical difference.

What the watermark actually does

Most production watermarks (Aaronson's Gumbel-softmax variant, Kirchenbauer's green-list bias) skew token probabilities at generation time. The detector then checks whether the observed n-gram distribution matches the expected biased distribution. It works statistically — give it 200+ tokens and you get p < 0.001. But humans don't read statistically. We read for coherence, voice, factual consistency. Those signals dominate.

The paper tested three conditions: unwatermarked LLM output, watermarked LLM output, and human-written controls. Participants saw all three in randomized order, no labels. Results:

• Watermarked LLM: 53% correctly identified as AI
• Unwatermarked LLM: 51% correctly identified as AI
• Human text: 68% correctly identified as human

The watermark moved the needle 2 percentage points. Noise.

Why detection fails at human scale

Two factors. First, the entropy reduction is subtle — typically 0.1-0.3 bits per token. That's below perceptual threshold. Second, modern instruction-tuned models already write with low perplexity on familiar topics. The watermark's "green list" tokens often coincide with high-probability tokens the model would pick anyway. The statistical signal exists but lives in the tail of the distribution humans never consciously access.

I ran a quick replication on a 7B Llama-3 variant with the standard KGW watermark (δ=2.0, γ=0.25). Detection AUC: 0.94 at 256 tokens. Human evaluators (n=12, CS grad students): 0.54. The gap is real.

# Quick detection script for KGW watermark
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

def detect_watermark(text, tokenizer, model, gamma=0.25, delta=2.0):
    tokens = tokenizer.encode(text, return_tensors="pt")[0]
    vocab_size = tokenizer.vocab_size
    green_list_size = int(gamma * vocab_size)
    
    z_scores = []
    for i in range(1, len(tokens)):
        prev_token = tokens[i-1].item()
        # Hash previous token to seed green list
        rng = torch.Generator()
        rng.manual_seed(prev_token)
        green_list = torch.randperm(vocab_size, generator=rng)[:green_list_size]
        
        current_token = tokens[i].item()
        in_green = current_token in green_list
        expected = gamma
        observed = 1.0 if in_green else 0.0
        z = (observed - expected) / (expected * (1 - expected))**0.5
        z_scores.append(z)
    
    return sum(z_scores) / len(z_scores) if z_scores else 0

Where this leaves detection

If watermarks don't help humans, we're back to classifier-based detectors — and those have their own problems (false positives on non-native English, brittleness to paraphrasing, adversarial evasion). The CMU paper suggests a pragmatic path: watermark for provenance (cryptographic audit trail), not human perception. Embed a verifiable signature in the generation log that downstream tools can check, accept that readers won't feel it.

Some labs are exploring semantic watermarks — biasing high-level structure (argument order, example selection) rather than token probabilities. Early results show slightly better human detectability (61% vs 53%) but at significant quality cost. Trade-offs everywhere.

The takeaway: don't ship watermarked output expecting users to "just know." Build detection into the platform layer where it belongs.