Stop making your stakeholders do mental math during eval reviews

If you have ever tried to present a performance breakdown of four different LLM candidates across ten different internal tools during a live meeting, you know the pain. You end up staring at a massive, cluttered dashboard, frantically reordering columns and filtering rows while your boss or investors stare blankly at a wall of numbers. It is impossible to communicate a clear trend when your audience has to perform cognitive gymnastics just to figure out which model is actually winning.

This is where heatmaps become a critical part of your AI workflow. Instead of forcing people to read line-by-line log values, a heatmap uses color intensity to represent numerical magnitude—like accuracy scores. The human eye is much better at spotting a bright yellow patch in a sea of purple than it is at comparing "0.87" vs "0.89" in a spreadsheet.

Moving beyond basic Inspect dashboards

While tools like inspect view are great for a quick deep dive into specific logs or sorting metrics to see model vs. model differences, they aren't built for high-level storytelling. If you are running complex evaluations with multiple independent and dependent variables, you need a 2D visual matrix to make the patterns jump out.

A well-constructed heatmap allows you to:

• Spot model ceilings: Instantly see where a model hits a performance plateau across different skills.
• Track skill gains: Visually confirm if adding a specific prompt engineering technique or tool-use skill actually moves the needle.
• Compare configurations: Line up models on one axis and skills on the other to see the entire performance spectrum at a single glance.

How to generate heatmaps with inspect_viz

To implement this in your own pipeline, you need the inspect_viz library installed alongside inspect_ai and pandas. I have been playing around with the implementation, and the core logic relies on the scores_heatmap function.

If you want to build this from scratch, you can look at the way the data is marshaled into the visualization. The script essentially takes your evaluation logs and wraps them into a horizontal or vertical orientation to make the axes readable.

Here is the basic command to run a heatmap generation script if you have your logs ready:

python3 inspect_viz_heatmap.py logs -o heatmap.html

Analyzing the output

When you generate the heatmap, you typically end up with a matrix where:

• The Y-axis (Vertical): Represents your models or specific configurations. This allows you to scan down a single column to see how a specific skill performs across different model versions.
• The X-axis (Horizontal): Represents the skills or test cases. Scanning across a row shows you exactly how a single model's performance fluctuates as the task complexity changes.

In a real-world deployment scenario, this is how you prove value. For example, if you see a high-intensity color (like a bright yellow) for a specific combination—say, 3.6-flash paired with a specific gemini_api_skill at a 1.0 score—it becomes an immediate talking point. You aren't just saying "the model is better"; you are showing a visual heat signature of success.

If you are building out a rigorous LLM agent evaluation framework, don't settle for raw logs. If you can't visualize the delta between your iterations, you aren't really seeing the full picture.