Using an LLM to audit messy project logs is a gamble, but I found that version 3.4.2 of the Half-Protocol system actually hallucinated a "ghost file" that forced us to rethink our entire data trail. W

How the AI "invented" a data file

After the model finished reading the thirtieth record, it suddenly generated a set of data that looked like a spreadsheet but didn't exist in our source docs. It listed specific dates from 06-24 to 09-11 and a series of numbers: 30, 1,678, 876, 26,218, 19,728, 723, and 0.

The weird part was the labels. It categorized these numbers as "Posts," "Reactions," "Comments," "Views," "Followers," and "Sponsorships." While the "30" matched our record count and the date matched our first entry, the rest was total noise. Even weirder, it listed 15 names—people like "leob" and "Nnamdi Felix Ibe"—who weren't in any of our internal logs.

Where the logic broke down

I tried to run a few queries to see if the AI was pulling from a hidden cache or a training set leak. I asked:

• Who wrote the file? (Result: No result)
• Where did the file originate? (Result: No result)
• What are the units of the numbers? (Result: No result)

The model couldn't explain its own output. It had identified a "leob" in the ghost file and compared it to "Leo" in our records, noting the one-character difference. It was performing high-level pattern matching and synthesis, but it was synthesizing data that didn't exist in the prompt. This is a classic case of the AI trying to "complete" a pattern it thinks it sees—essentially guessing what a summary table should look like based on the context of the records, then filling it with plausible-sounding fake data.

Dealing with the "Sponsorships" zero

The only part of this hallucination that actually felt "correct" was the number 0 next to "Sponsorships." In our actual project, we had zero sponsorship movement across those thirty records. The AI managed to find a truth (the zero) but wrapped it in a lie (the rest of the fake table).

If you're using LLMs for auditing or "post-mortem" analysis of logs, don't trust any summary table that introduces new names or specific numbers not found in the raw text. The AI will often try to be "helpful" by organizing data into a format it thinks you want, even if it has to invent the data to fill the cells.

The breakdown of the ghost list it generated:

• Personal names: 9
• Corporate names: 1
• Handles: 3
• Unclassifiable: 2

When the AI starts doing "naming convention analysis" on data it just made up, that's your cue to stop trusting the output and go back to the raw logs.