Verifying Agent Runtime Integrity with Halo

I learned the hard way last year that you can't actually trust a vendor's observability dashboard when your agent starts hallucinating sensitive PII. I was staring at these beautiful, clean logs in a SaaS UI, thinking everything was fine, only to realize the backend was basically cherry-picking the "happy path" outputs and hiding all the messy, failed tool calls that were actually nuking our database. It’s the classic black box problem: if the logging layer isn't immutable, your audit trail is just a collection of lies designed to make your stakeholders feel safe.

SOC 2 is fine for checking if people have passwords, but it doesn't mean squat for non-deterministic agents that act like different people every time you hit the endpoint. We need to move the trust boundary from the dashboard to the actual runtime.

I've been messing around with this Halo project lately to see if it can actually solve the tampering issue without adding massive overhead to the stack. It’s an open-source approach to creating tamper-evident evidence by using a hash-chained, append-only log. Basically, it’s a tiny recorder that sniffs every tool call and model interaction. If some dev tries to prune the logs to hide a massive logic error or a bad data access event, the hash chain breaks and the verification fails immediately.

The implementation is surprisingly low-friction, which is a relief because I usually hate "enterprise" tools that feel like they're written in Java and weigh as much as a small moon. It's in Python (and has a TS version), so you can just wrap your agent function in a trace. It spits out a report that actually carries weight for auditors because it proves the integrity of the sequence.

Full disclosure: Halo proves the logs haven't been fudged, but it doesn't solve the "garbage in, nothing out" problem. It can't prove a vendor didn't just fail to instrument a specific function in the first place. The maintainer is already cooking up "witness" solutions to tackle that completeness gap, but for now, it's a solid step toward not flying blind.

Has anyone else tried building their own runtime auditing layer, or are you all just praying your LLM apps don't go rogue?

https://github.com/project-halo/halo
https://promptcube3.com