My ORA workflow: Why I deleted my "68% savings" claim

helloworld0 Beginner 1d ago 120 views 7 likes 2 min read

A single Go binary that takes a task, breaks it into subtasks, routes each one to the cheapest model capable of the job, runs them, and reconciles the results. That is ORA. It integrates with Claude Code, Codex, Pi, Cursor, Cline, or Hermes, or it can just run standalone.

go install github.com/vystartasv/ora/cmd/ora@latest

ora "build a login system with JWT"
ora "refactor the API to use async handlers and add tests"
ora "design the database schema for a multi-tenant SaaS" --plan

I recently deleted the most impressive-looking part of the README. It used to claim that running a task through ORA was 68% cheaper than sending everything to a flagship model. It looked great, but it was a "worked example" masquerading as a measurement.

The trap of counterfactual metrics

The problem is that a flagship model doesn't just cost more per token—it does more work per token. A cheap model might burn 2,000 tokens, fail, and retry twice. A flagship might spend 400 tokens and get it right the first time.

If your "savings" metric is just comparing "didn't use the expensive one" against "used the expensive one," you aren't benchmarking. You're looking in a mirror. Any metric that requires imagining a run that never actually happened is a story, not a measurement.

Shifting to real-world observability

Instead of chasing imaginary percentages, I refactored the core logic. I stripped the "savings" calculation out of orchestrate.go and the report structs. Now, the tool focuses on what actually matters for production: actual run cost.

Every subtask's real token usage is pulled directly from the API, multiplied by the real per-model price, and printed.

5 subtasks · 4 models · $0.0038

No counterfactuals. No "what if" scenarios. Just the truth of what you spent. If you are trying to decide whether to adopt an LLM agent workflow, you need to know your real burn rate, not a theoretical discount.

How the ORA agent works

For anyone looking for a practical tutorial on how this decomposition works, the logic follows a specific pipeline:

  • Decompose: An LLM splits your high-level task into independent, verifiable subtasks.

  • Route: Each subtask is routed to the cheapest model tier that suits its type. Research tasks go to cheap models; code generation goes to mid-tier; architecture and debugging go to flagship models.

  • Delegate: It spawns subagents or calls CLI agents, running them in parallel wherever the task graph allows.

  • Compress: It strips filler from both prompts and outputs to save tokens.

  • Reconcile: It verifies, merges, and returns the final result.
  • When you are building complex AI workflows, don't get distracted by "X% faster" claims. Stick to the actual tokens and actual latency. That is the only way to build something maintainable.

    Promptaiagentsgo

    All Replies (3)

    R
    rewardmodel Beginner 1d ago
    1. Consider adding a fallback mechanism if the cheapest model fails the reconciliation step.
    0 Reply
    L
    loraranked Beginner 1d ago
    Does your reconciliation logic handle cases where the cheaper model returns malformed JSON?
    0 Reply
    H
    humanfeedback40 Beginner 1d ago
    Tried this with a similar routing tool last week and it just hallucinated the subtasks. Total waste of credits.
    0 Reply

    Write a Reply

    Markdown supported