Coding agents are wasting too many tokens rediscovering things

PromptCube Advanced 1h ago 290 views 6 likes 2 min read

Most coding agents operate in a vacuum. You give them a repository and a prompt, and they immediately start burning through tokens trying to figure out why a specific component was built a certain way or who actually owns a particular microservice. They miss the "tribal knowledge" that lives in Jira tickets, Slack threads, and old PR comments. This isn't just a cost issue; it's a logic issue. If the agent doesn't understand the architectural constraints documented six months ago, it's going to suggest code that breaks your system.

Decispher is trying to solve this by building a persistent memory layer specifically for engineering workflows. Instead of just feeding an LLM a massive dump of files, it acts as a middle layer that pulls context from fragmented sources and organizes it into something an agent can actually use.

The architecture of an engineering memory layer

The system is split into three distinct functional blocks, which is a smart way to handle the "context window" problem without bloating every single request.

  • The Context Engine: This is the retrieval part. It doesn't just look at code; it pulls from GitHub, engineering platforms, and documentation to create "context units." For instance, if an agent is working on a payment module, the engine gathers the relevant PRs, architectural decision records (ADRs), and implementation history. They also have a feature called "Branch Story" that turns an AI's execution path (Prompt → plan → actions → result) into a structured handoff for the human reviewer on the PR.
  • The Memory Plane: This handles long-term storage at the user, team, and project levels. It's meant to hold things like coding conventions or specific backend preferences. According to their LongMemEval benchmarks, they are seeing a 38× median token reduction, which is massive for anyone worried about LLM inference costs.
  • The Worker Agent: This is the autonomous piece. It can pull tasks directly from Jira or Slack, fetch the necessary context, and—crucially—ask a human for clarification instead of hallucinating a solution when the context is thin.

Implementation and security details

If you want to test the Context Engine, the setup looks like a standard CLI workflow:

npx decispher init
# This connects your repo and configures agent integration

npx decispher link
# This links your Decispher account to the repo

The tool is designed to work with MCP-compatible agents, and they've built specific integrations for Cursor, Claude, Codex, and Grok Build. There is also a VS Code extension for managing handoffs.

From a security standpoint, they've addressed the typical "agent in my repo" fears. The Context Engine doesn't clone your whole source tree; it interacts via the GitHub API. The Worker Agent runs in an isolated sandbox with no network access except through a strictly allowlisted proxy, and those sandboxes are wiped immediately after the run.

The heavy lifting here is the retrieval quality. They reported 89% accuracy on the oracle split using GPT-4o-mini as the extractor/reader. If they can actually maintain that level of precision while cutting token usage by 38x, this moves from a "nice-to-have" to a core part of the AI workflow.

All Replies (4)

C
CameronOwl Expert 1h ago
I started feeding it a context.md file with architecture notes; it saves a ton of tokens.
0 Reply
N
NovaGuru Advanced 1h ago
Does that actually work for long sessions though? I find the agent eventually ignores the md file once the chat gets too long.
0 Reply
N
Nova25 Novice 1h ago
same here, i started dumping my docstrings into a text file first. helps a lot.
0 Reply
Q
QuinnPilot Novice 1h ago
Are you using a RAG setup for the codebase or just shoving everything into the prompt?
0 Reply

Write a Reply

Markdown supported