LLMs are basically blind to what isn't there in clinical notes

PromptCube Intermediate 1h ago 45 views 11 likes 3 min read

If you ask an LLM to grade a medical note, it will congratulate you on the stuff you wrote perfectly while completely ignoring the fact that you forgot to mention the patient's life-threatening allergy. It turns out that using an LLM as a judge for clinical documentation is like hiring a critic who only notices the ingredients present in a soup but fails to realize the chef forgot the salt entirely. This "omission blindness" is a massive problem for anyone trying to build a reliable AI workflow in healthcare.

The core issue is that Large Language Models are trained to recognize patterns and confirm existing information. When we use them to verify clinical accuracy, they excel at "presence verification." They see a diagnosis, they see a medication, and they tick a box. But they are remarkably bad at "absence verification." They can't easily verify the non-existence of a symptom or the omission of a mandatory follow-up instruction because, well, there's nothing there to process.

The mechanics of the failure

When we run a deep dive into how these models handle clinical reasoning, the pattern is consistent. An LLM judge evaluates a note based on a checklist. If the checklist says "Verify if the patient's blood pressure is recorded," the LLM looks for numbers. If it finds them, it passes the note.

However, if the checklist requires "Ensure no contraindications to the prescribed dosage are missed," the LLM struggles. It looks at the note, sees the dosage, and thinks, "Looks good to me!" It doesn't perform the internal reasoning loop required to check the absence of a warning that should have been there. It lacks the "negative reasoning" capability required for high-stakes medical auditing.

Why this ruins your AI agent deployment

If you are building an LLM agent to automate medical scribing or auditing, this isn't just a minor bug; it's a liability. You might think your system is performing with 99% accuracy because the LLM judge is giving everything a thumbs up. But that accuracy is a lie. It's only measuring how well the AI can repeat what it sees, not how well it can identify what is missing.

To fix this, we can't just rely on a single prompt. A simple prompt engineering approach won't solve a fundamental architectural weakness in how transformers process tokens. You can't "find" a token that doesn't exist.

A better way to build the workflow

If you want to actually catch omissions, you have to change your deployment strategy. Instead of asking one LLM to "judge the note," you need a multi-step process:

1. Extraction Phase: Use a specialized model to extract every single clinical entity actually present in the text.
2. Knowledge Retrieval: Pull the required clinical standards or "gold standard" checklists for that specific patient scenario from a structured database.
3. Comparison Phase: Instead of asking the LLM to "check the note," you feed the difference between the extracted entities and the required checklist into the model.

Essentially, you have to do the heavy lifting of finding the gap before you ask the LLM to comment on it. You have to turn an "absence" problem into a "presence" problem. Until we solve this, using LLMs as autonomous clinical auditors is like asking a person to proofread a book by only looking at the words that are actually printed on the page.

Medical AIClinical Notes

All Replies (3)

C
CameronCat Intermediate 1h ago
True. It also tends to hallucinate details to fill those gaps if you don't prompt it carefully.
0 Reply
C
ChrisPunk Novice 1h ago
Does this happen even with few-shot prompting or is it just a fundamental architecture issue?
0 Reply
S
SoloSage Advanced 1h ago
I've seen this too; it missed a whole section on medication changes in my last audit.
0 Reply

Write a Reply

Markdown supported