RLHF is not enough to keep autonomous agents from wrecking your

Training a model with RLHF or DPO to be "safe" is basically just teaching it to be polite. That works fine for a chatbot, but it's structurally insufficient for an LLM agent that has the keys to your file system or API endpoints. If an agent can execute code and mutate databases, safety cannot be a "vibe" instilled during training—it has to be a runtime contract enforced by the harness.

The two sides of the safety contract

When we talk about runtime contracts, we aren't talking about more prompt engineering. We are talking about a hard boundary between the model's intent and the system's execution. I see this as having two distinct faces:

• The Preventive Face: This is the traditional "guardrail" approach but moved to the infrastructure level. Think sandboxes, strict permission gates, and trajectory monitors that kill a process the moment it deviates from a safe path. It's about blocking the disaster before the rm -rf command actually hits the disk.
• The Evidential Face: This is where most current AI workflows fail. Instead of trusting the agent when it says "I've fixed the bug," the system should require verifiable proof. Task submission should be gated by hard evidence—file diffs, successful test run logs, or grounding citations. If the agent can't produce the receipt, the action isn't committed.

Why training-time safety is a losing battle

There is a massive imbalance in how the industry views this. If you look at the research coming out of NeurIPS or ICML, there's an 8x to 12x tilt toward training-time safety over deployment-time safety. But the real-world data tells a different story. Looking at documented agent incidents, the failures aren't usually because the model "forgot" its safety training; it's because the agentic loop allowed a hallucinated or erroneous action to execute without a check.

We've seen this movie before in computer security and experimental science. Both fields realized that you can't just "train" a system to be safe; you need a runtime environment that enforces constraints and demands evidence.

Moving toward a Trajectory Schema

The real unit of safety in an AI workflow isn't the model—it's the trajectory combined with checkable evidence. We need to stop asking "Is this model safe?" and start asking "Is this trajectory verifiable?"

Implementing this requires a formal Agent Trajectory Schema. Instead of a raw stream of tokens, the agent's path should be a series of compositional gates. Each step must be monitored, and each state change must be backed by an evidence chain. This shifts the burden of safety from the probabilistic nature of the LLM to the deterministic nature of the harness.

For anyone building a real-world LLM agent, the goal should be a deployment where the model is treated as an untrusted actor and the harness acts as the sole source of truth.