Bullet is hitting 95.8% on SWE-bench Verified and it's way faster

Reducing round trips between the model and the environment is actually more important than the raw tokens-per-second of the LLM itself. This is the core realization behind Bullet, a coding agent that manages to resolve 479/500 tasks on SWE-bench Verified in one attempt, averaging just 119 seconds per task. When compared to setups like mini-SWE-agent with Fable or Sol, Bullet is running 35–67% faster depending on the complexity of the task.

The founders basically built this because they were sick of the latency and inefficiency of existing tools like Claude Code. Instead of just hoping for a faster model, they focused on the AI workflow and how the agent actually interacts with the codebase.

How they actually optimized the agent loop

If you're into prompt engineering or building LLM agents, the technical choices they made to kill latency are worth noting:

1. Smart Model Routing: They stopped blindly sending every task to the heaviest model. If Sonnet can handle it, it goes to Sonnet, preventing unnecessary compute lag on simpler tasks.
2. Targeted Search over Embeddings: Rather than embedding the entire repository—which can be noisy—or stuffing a massive compressed context into the window, they implemented high-speed, targeted greps to find exactly what's needed.
3. Context Hygiene: This is a huge one for avoiding "model drift." They bound tool outputs and purge stale screenshots or redundant file reads so the context window doesn't get flooded with garbage.
4. Parallel Investigation: Instead of a linear "search -> read -> edit -> verify" loop, Bullet batches independent investigations. It gathers all necessary info in parallel, makes one surgical edit, and then does a single focused verification. This reportedly cut round trips by 16% and costs by 27%.

Real-world friction and findings

The development process revealed some annoying "silent killers" in AI coding. For instance, regex-dialect mismatches in code search can lead an agent down a completely wrong path without the model even realizing it missed the target. To fix this, they had to build a search system with specific fallbacks and bounded context to ensure the agent stays on track.

The most practical use case for this kind of speed isn't just small bug fixes, but long iterative workflows—things like data pipelines or evaluation loops where each step relies on the previous one. In those scenarios, you can't just spin up ten parallel agents; you need a single agent that can move through the sequence as fast as humanly possible.

If you want to see the specific benchmarks or the methodology they used to hit those numbers, they've detailed it here:

https://www.codewithbullet.com/blog/benchmark-results.html

For a visual on how the agent handles a live codebase, there's a demo available:

https://www.youtube.com/watch?v=rWVmG5fRKgE