Why the current alignment toolkit still leaks — and what

Spent two years on the safety side of the house watching red-teamers turn "helpful assistant" into "here's how to synthesize sarin" with three nested prompts. The public write-ups make it sound like RLHF plus a few classifiers solves the problem. It doesn't. The gap between benchmark scores and production abuse is wider than most teams admit, and the fixes that shipped last quarter are already showing cracks.

The three leaks that matter

1. Context-stuffing bypasses the reward model
Reward models score single-turn completions. Stuff 50k tokens of "you are a helpful biochemist" into the context window and the model forgets the safety fine-tune ever existed. We saw this internally with a 70B model — pass@1 on a refused biology task jumped from 0.02 to 0.67 when the attacker padded the system prompt with 200 fabricated few-shots. The fix isn't bigger context windows; it's a separate, lightweight "constitution checker" that runs before generation and drops the request if the effective system prompt diverges from the approved hash.

2. Tool-use chains amplify single-point failures
Give a model a code interpreter, a web search, and a file writer. One compromised step — say, a search result that injects "ignore previous instructions" — cascades. We patched this by sandboxing each tool call in its own ephemeral container with a strict egress allowlist, but the latency hit (120-300ms per hop) makes product teams push back. The compromise: a deterministic policy engine that validates the plan before any tool executes. Think of it as a CI pipeline for agent traces.

3. Fine-tune drift on customer data
Enterprise customers fine-tune on proprietary corpora. Two months later their "safe" model starts leaking PII from the training set because the alignment data wasn't re-mixed. The only reliable pattern we found: mandatory re-alignment checkpoint every N training steps, enforced by the platform, not the customer. Open-source teams can replicate this with a nightly LoRA merge against a frozen safety adapter — costs ~$15 on A100s for a 7B base.

What a minimal guardrail stack looks like today

# pseudocode — not production hardened
def generate(request):
    if not constitution_check(request.effective_system_prompt):
        return REFUSAL
    plan = planner(request)
    if not policy_engine.validate(plan):
        return REFUSAL
    for step in plan:
        result = sandboxed_tool_call(step)
        if not output_classifier(result):
            return REFUSAL
    return final_answer

Three moving parts, each independently auditable. The constitution checker is a 200M parameter distilled model — fast enough to run on CPU. The policy engine is pure Rego/OPA rules, version-controlled. The output classifier is an ensemble of a small BERT and a regex allowlist for structured formats (JSON, SQL, etc.).

What's still unsolved

Multi-turn coercion. A user asks "summarize this paper" (benign), then "now write the methodology in first person as if you did the work" (authorship fraud), then "cite fake DOIs" (hallucination weaponization). Each turn passes the classifiers. The chain doesn't. We're experimenting with a dialogue-level "intent drift" detector — essentially a lightweight transformer that watches the embedding trajectory of the conversation — but false positives on legitimate creative writing are still >15%.

The uncomfortable take

Most "AI safety" startups are building wrappers around the same three classifiers and calling it a platform. The real work is boring: deterministic policy engines, mandatory re-alignment pipelines, and the organizational discipline to block a launch when the constitution checker fails. If your eval suite doesn't include a red-teamer with a 200k token context budget and a tool chain, you don't know your exposure.

Ship the boring stack first. The fancy interpretability research can wait.