Echo: Getting Fable-level Results with Open-Weight Models

Using a pool of open-weight models instead of betting everything on one "super-model" is surprisingly efficient. Echo is basically an experiment in dynamic allocation—instead of picking one LLM for every task, it decides on the fly which models to engage and how to merge their outputs based on the request.

The logic is pretty straightforward: some prompts are trivial and only need a tiny bit of inference, while others are complex enough to require multiple models tackling different angles of the problem. The wild part is how complementary these models are. A model that fails miserably on a general benchmark might be the absolute MVP for a specific niche problem. When you combine them correctly, the aggregate performance beats any single model in the pool.

In terms of the actual numbers, this setup hit roughly the same results as Fable but at about 1/3 of the cost. It's basically a way to optimize the AI workflow by treating models as specialized tools rather than general-purpose replacements for each other.

The system isn't perfect—it still makes the wrong call on which model to allocate occasionally—but the cost-to-performance ratio is hard to ignore. For anyone looking for a real-world example of an LLM agent orchestration layer, this is a great case study.

If you want to test the deployment, there's a chat interface and an OpenAI-compatible API available.

https://echo.tracerml.ai/

For those who like to dig into the data, the evaluation methodology and cost breakdowns are detailed here:

https://echo.tracerml.ai/eval