LLM watermarking isn't about visible text or hidden ads

Most people assume that when a company like Anthropic talks about "watermarking" their model responses, they mean some sort of digital signature or a weirdly placed advertisement tucked into the prose. I used to think the same way—that there would be some detectable, visible pattern. But after digging into the technical side, it turns out a watermark in a Large Language Model (LLM) is actually a subtle statistical shift applied during the token selection process. It’s not something a human reads; it’s something a detector calculates.

I decided to build a minimal, educational version of a SynthID-Text style watermarking system to see if I could actually wrap my head around the math. It isn't a 1:1 replica of Google's proprietary SynthID system, but the core logic of how you manipulate token probabilities to embed a signal is all there.

The core concept: Red/Green lists

The fundamental trick involves splitting the vocabulary into two sets—let's call them "green" and "red" lists—based on a pseudo-random hash of the previous token. When the model is deciding which token to generate next, you artificially boost the probability of tokens from the "green" list.

If a piece of text is purely human-written, the distribution of tokens should be relatively "natural." However, if the text was generated by a watermarked model, you will see a statistically impossible concentration of "green" tokens. A detector can then run a hypothesis test to see if the frequency of these green tokens deviates significantly from what we'd expect by chance.

A simplified implementation approach

To make this work in a practical tutorial style, you basically need to intercept the logits (the raw scores before softmax) during the inference step. Here is a high-level breakdown of the workflow:

1. Generate a pseudo-random sequence: Use the previous token's ID as a seed for a hash function to determine which tokens in the current vocabulary belong to the "green" list.
2. Modify the logits: Add a small constant (a "bias" value) to the logits of all tokens in the green list.
3. Sample the token: Perform the standard softmax and sampling. Because the green tokens now have higher scores, the model is much more likely to pick them.
4. Detection: To verify, you don't look at the words; you look at the ratio of green tokens to red tokens across a long string of text.

I put together a repository that demonstrates this logic from scratch. It’s a great way to understand the tension between "watermark strength" (how easy it is to detect) and "model utility" (how much the watermark ruins the actual quality of the text). If you push the bias too high, the model starts talking nonsense; if you keep it too low, the watermark is too easy to strip away with simple rephrasing.

If you want to poke around the code and see the actual math in action, you can find the implementation here:

https://github.com/Saad1926Q/llm-watermark