World-Model-Optimizer: Cutting LLM Costs via Distillation

Running high-end frontier models for every single agent task is a massive waste of budget when 80% of the queries are repetitive. We've been looking for ways to trim our API spend without sacrificing quality, and the wmo serve tool hits exactly that pain point by routing tasks to smaller, distilled models.

The core logic is simple: it uses your existing agent traces to identify patterns. Instead of hitting a frontier model every time, it distills that knowledge into a smaller model and routes the repetitive stuff there.

Here is how the workflow actually functions:

1. Trace Capture: It takes the logs/traces your agents are already generating.
2. Distillation: It uses those traces to refine a specialized smaller model.
3. Routing: A router decides in real-time if a request needs the "big" model or if the custom distilled model can handle it.
4. Compaction: It strips out noise to reduce token usage.

For a local setup, you just provide the traces and an OpenRouter key. It spins up an OpenAI-compatible endpoint locally. As more traces come in, the system keeps training, meaning the smaller model gets more accurate over time.

From a deployment perspective, this is a practical way to build a production AI workflow where you aren't paying "frontier prices" for basic tasks. The claim is a 40%+ reduction in costs while maintaining equivalent quality.

If you want to check out the demo, the link is here:

https://www.youtube.com/watch?v=2_m4Ze6mdko

For those who don't want to manage the infrastructure, there is a hosted version available via their waitlist:

https://experientiallabs.ai