Visualizing the internal workspace of LLMs

I learned the hard way that trusting a model's output without understanding its internal state is a recipe for debugging nightmares (a lesson I paid for in broken production pipelines). Anthropic’s work on "Verbalizable Representations" actually provides a way to look into those middle layers using a Jacobian Lens, and someone actually had the audacity to port this concept to open-source models.

Testing this out with prompts like "three curving lines of water" yields some surreal results. You can literally watch tokens like "ocean" or "sea" activate in the deeper layers before the model finally settles on "waves." It’s not just passive observation, either. You can manually inject a concept—say, "fire"—directly into the middle layer while it's processing the water prompt, and the output shifts toward heat-related descriptions in real-time. It’s essentially a way to hijack the model's internal direction.

The most interesting part for anyone concerned with developer experience and model reliability is the implementation of a way for the model to read its own workspace. It can essentially suppress or amplify specific concepts, which feels like a much more practical path toward alignment than just feeding it more Reinforcement Learning from Human Feedback (RLHF) and hoping for the best.

From a technical standpoint, the architecture performance is surprisingly inconsistent (which is usually how these things go). Probing a 0.5B Qwen model actually yielded better readability than a 2.8B Pythia. It seems Llama and Qwen architectures are significantly more responsive to this kind of probing than others, which might save us some headache when choosing which models to actually bother with for interpretability work. It is a remarkably sophisticated setup for a project that has only been live for a few days.

https://transformer-circuits.pub/2026/workspace/index.html

https://lucid.earthpilot.ai/research