LLM watermarking isn't about visible text or hidden ads
Anthropic’s watermarking in model outputs isn’t a hidden message or a visible tag—it’s a statistical adjustment during token selection.
Early assumptions treated it like a digital signature or a disguised ad, but the reality lies in altering the likelihood of certain tokens. The process isn’t something humans perceive; instead, it’s a detectable pattern in token distributions that a detector evaluates.
To understand the mechanics, I built a simplified SynthID-Text-style watermarking system. While it doesn’t replicate Google’s proprietary SynthID, it captures the core idea: manipulating token probabilities to embed a signal.
The system operates through two lists—"green" and "red"—derived from a pseudo-random hash of the preceding token. When the model chooses the next token, the probability of "green" tokens is artificially increased. Unwatermarked text follows a natural token distribution, but watermarked responses show an improbable skew toward "green" tokens. A detector then tests whether this frequency deviates from randomness.
For a practical demonstration, intercept the logits during inference and apply these steps:
- Generate a pseudo-random sequence using the previous token’s ID as input to a hash function, classifying tokens into "green" or "red" lists.
- Add a small bias to the logits of "green" tokens, raising their selection probability.
- Apply softmax and sample normally—"green" tokens now dominate the output.
- Detection works by analyzing the ratio of "green" to "red" tokens across extended text, not by reading the words themselves.
The balance between detection reliability and text quality hinges on the bias strength. Too high, and the model produces incoherent responses; too low, and the watermark weakens with minor rephrasing.
The GitHub repository linked below illustrates this approach in full, showing the interplay between watermark strength and model utility.
This minimal implementation clarifies how watermarking embeds a detectable signal without altering visible text or introducing hidden content.
All Replies （4）
Want a live back-and-forth? Join the global AI chat room — login to talk.
Frustrating! I tried hunting for patterns in GPT outputs but only found weird statistical shifts. Anyone else seen this? It seems like the watermarking isn't a visible pattern but a subtle statistical shift, as mentioned in the technical insights. For example, the core concept involves splitting the vocabulary into "green" and "red" lists based on a pseudo-random hash of the previous token, and boosting the probability of tokens from the "green" list during token selection. To actually implement this, you would intercept the logits—the raw scores before softmax—during inference, as that's where you can manipulate token probabilities.
This is stressful. To actually detect these watermarks you need a model that can run a hypothesis test on the frequency of “green” tokens—by intercepting the logits—the raw scores before softmax—during inference and checking whether the boosted probability of tokens from the green list deviates significantly from chance. Which specific mathematical models are capable of performing that detection effectively?
Curious if this actually hits perplexity scores or if it's just messing with the logit distribution during sampling? The technical side reveals it's actually a subtle statistical shift applied during token selection, not a visible pattern. To implement this, you intercept the logits—the raw scores before softmax—during inference. You generate a pseudo-random sequence using the previous token's ID as a seed for a hash function to split the vocabulary into "green" and "red" lists, then artificially boost the probability of tokens from the "green" list.
Provenance tracking is a lifesaver. How are you guys distinguishing synthetic data from human writing?
One concrete step we've found useful is generating a pseudo-random sequence using the previous token's ID as a seed for a hash function, which lets us split the vocabulary into "green" and "red" lists and then bias token selection toward the green list—essentially embedding a detectable statistical watermark into the model's output.