World-Model-Optimizer: Cutting LLM Costs via Distillation
world-model-optimizer (WMO), and it tackles this by using agent traces to distill specialized, smaller models that handle the boring stuff while the heavy hitters only kick in when necessary.The core logic here is a routing system. Instead of a static prompt, wmo serve acts as an intelligent gateway. It analyzes incoming tasks; if a task matches a pattern the distilled model has already mastered via previous traces, it routes it there. If it's something complex or new, it hits the frontier model.
For those wanting to set this up as a practical tutorial for their own AI workflow, the process is straightforward:
1. Capture your agent traces (the logs of how your frontier model successfully solved tasks).
2. Provide those traces and an OpenRouter API key to WMO.
3. Deploy the wmo serve endpoint.
The system then handles the token compaction to strip out noise and continuously trains the smaller model as more data flows in. It's essentially a self-improving loop for LLM agent deployment.
- Cost reduction: 40%+ lower spend on tokens.
- Latency: Faster responses by utilizing smaller, distilled models for repetitive tasks.
- Optimization: Token compaction removes unnecessary fluff from prompts.
If you're running a production agent that handles thousands of similar queries, this is a much smarter approach than just hoping for a price drop from the model providers. It turns your own usage data into a proprietary performance advantage.
https://www.youtube.com/watch?v=2_m4Ze6mdko