Mapping messy intent to deterministic technical specs

I learned the hard way during a previous production rollout that letting an LLM act as a black box for query parsing is a recipe for data leakage and catastrophic schema mismatches. When I started building this recommendation engine to map natural language queries against a database of 13,000 models, I wasn't interested in "vibes." I needed a way to extract semantic intent without losing the technical rigor required for satellite imagery scanning or specific model parameters. I opted for a hybrid retrieval setup using spaCy because, in my experience, you need that layer of control to ensure your extraction logic doesn't drift into hallucinations.

To stress-test the architecture, I ran a highly controlled experiment that honestly felt like a liability nightmare waiting to happen. I instantiated three AI agents as "Lead Research Scientists," but I sandboxed them heavily—no regex, no unvetted libraries, just Python built-ins and spaCy. I gave them a custom-prepared vocabulary and full agency to spin up sub-agents and web tools to simulate a proper laboratory environment. They were tasked with building their own intent-inference engine based on my constraints.

The real friction started when I moved from execution to validation. Instead of running a standard unit test suite, I orchestrated what I can only call an "adversarial peer review." I brought in a panel of high-end models, specifically Claude and Codex, and instructed them to act as ruthless judges. My prompt was simple: find the lies, tear apart the logic, and debate the scientific validity of the outputs. I expected a dry, technical log of errors. Instead, I witnessed a high-entropy intellectual brawl. The judges were relentless, cross-examining the "scientists" and eventually turning on one another when the consensus failed to materialize.

After hours of the agents debating the held-out test sets, the final output from the LLM panel was unsettlingly meta: "Nobody won. We need a human." It was a stark reminder that even with sophisticated agentic workflows, there is a massive gap between simulated reasoning and actual human-level verification. If you want to audit the actual logs and the raw JSON transcripts—which are about a megabyte of pure chaos—you can find the full repository here:

https://github.com/ (Note: You'll need to find the specific repo for the intent-inference experiment)

And if you're looking for the interface I used to manage the agentic orchestration, check out:
https://promptcube3.com