Artificial Analysis Search Index benchmark highlights Parallel, Exa, and Firecrawl

Artificial Analysis has released its Search Index benchmark, and the findings align with production observations where search feeds agent loops, making Parallel, Exa, and Firecrawl the three strongest foundations.

How the Search Index benchmark was run

The evaluation put seven providers through GPT-5.6 Luna, measuring quality, cost, and latency. Many comparisons focus only on relevance scores, but this benchmark assigns weight to all three factors that matter to agents: whether the model can trust snippets, whether costs remain manageable at scale, and whether each round trip completes before users grow impatient.

Why Parallel claimed the quality lead

Parallel claimed the quality lead. Its results repeatedly surfaced the most citation-worthy sources while introducing the least hallucination-inducing noise. That advantage is especially important for research-heavy agents synthesizing material across 20+ sources. Exa occupied the speed-cost frontier, delivering sub-800ms p95 and pricing that remains reasonable even with deep pagination. Firecrawl was the surprise: its extraction quality stayed competitive with bigger names, while it was also the only option offering a generous free tier that supports prototyping without a credit card.

Fatal flaws in the also-ran providers

The also-rans — Tavily, Serper, Bing, and Google Custom Search — each had a fatal flaw. Tavily's latency spiked past 2s on complex queries. Serper's snippet quality degraded noticeably past page 2. Bing and Google CSE both impose rate limits that break parallel agent fan-out patterns unless you negotiate enterprise deals.

The benchmark doesn't capture API ergonomics. Parallel's streaming response format lets you start parsing before the full payload lands. Exa's category filter (news, academic, github, etc.) saves a ton of post-filtering logic. Firecrawl's markdown output is cleaner than anything else — zero regex cleanup needed before feeding into the context window.

When Exa's category routing wins

If building a coding agent that needs GitHub issues and docs, Exa's category routing is a force multiplier. For general research agents, Parallel's quality edge compounds over multi-hop reasoning. For side projects and MVPs, Firecrawl's free tier removes the "but what if this costs $200/month" hesitation.

Two production pipelines were migrated to Parallel + Exa hybrid routing based on query type. The quality delta showed up immediately in eval scores — fewer "I couldn't find reliable sources" failures, more complete answers on the first pass.

Worth pulling the full report when evaluating. The raw latency distributions and cost-per-1k-queries tables saved a week of A/B testing.