Browser Tools SDK: A Real-World Agent Benchmark

A 24/26 pass rate on live-site tasks is a bold claim, but the cost efficiency is what actually catches my eye. I've been skeptical of "browser-enabled" agents because they usually burn through tokens by dumping the entire DOM into the context window. The Browser Tools SDK claims to be 55% cheaper than alternatives like playwright-cli or agent-browser, which suggests they've actually solved the context bloat problem.

The architecture is lean—only 6 tools in total. The real heavy lifting happens via browser_snapshot (for orientation) and browser_exec (for raw Playwright execution). Relying on the model's pre-existing knowledge of Playwright is a smart move; it's better than trying to map complex browser actions to a limited set of custom functions.

If you're building an LLM agent and need a production-ready harness, this is a practical tutorial in minimalism. Here is how the implementation looks:

import { createAiSdkBrowserTools } from "libretto-browser-tools/ai-sdk";
import { LocalBrowserProvider } from "libretto-browser-tools";

const { tools } = createAiSdkBrowserTools(new LocalBrowserProvider());

const result = await generateText({
  model: anthropic("claude-sonnet-4-5"),
  tools,
  prompt: "Go to Hacker News and tell me the top story",
});

Comparing this to other browser wrappers:

• Pass Rate: Tied for best (24/26 tasks)
• Cost per Task: $0.106 (vs $0.235 for the nearest competitor)
• Token Usage: 1.45M tokens (vs 2.29M+ for alternatives)

It supports various infra providers including Libretto Cloud and Browserbase, or you can stick with LocalBrowserProvider for local Chromium runs, though you'll likely hit more bot detection walls that way. For anyone doing a deep dive into AI workflow automation, reducing token overhead is the only way to make these agents viable at scale.

The documentation is available here:
https://libretto.sh/docs/browser-tools/quickstart