Scraping costs are spiraling out of control

Raw HTML is a silent killer for agentic workflows. When an LLM tries to parse a standard Wikipedia article, it's pulling in 68,240 tokens just to read the text. If you attempt to hit a commercial site like Nike, you're looking at 353,000 tokens of junk data. That isn't just a cost issue; it's a reliability and context window issue.

The current state of agentic web interaction is broken for three reasons:

1. Token Bloat: Agents often dump massive, unparsed HTML blocks into the context window when they fail to find specific data, essentially burning money to tell you nothing.
2. Fragile Fetching: Built-in tools like webfetch work for static summaries, but they crumble against JS-rendered pages or basic anti-bot measures.
3. Hallucinations: When an agent hits a 403 Forbidden or a blank page, it doesn't always stop; it often starts hallucinating based on stale training data because it lacks a reliable live view.

I've been experimenting with a different approach to this workflow. Instead of letting the agent struggle with raw DOM trees, I built an MCP (Model Context Protocol) server using a recompiled Chromium instance designed for stealth.

The goal wasn't just to bypass blocks, but to transform the data pipeline. Instead of getting a wall of unreadable code, the MCP returns cleaned, lightweight tokens. A task that previously failed or returned massive HTML now returns around 700 tokens for a site like Nike, or just 285 tokens for specific JS-heavy elements.

It’s not a silver bullet for heavy Kasada-style protections, and it lacks residential egress, but for QA and research automation, it keeps the context window clean.

https://github.com/tiliondev/fortress/tree/main/mcp