AI content is detectable with 98% accuracy using structure alone

Riley82 Advanced 1h ago 501 views 3 likes 2 min read

The structural "shape" of a blog post is a dead giveaway for AI authorship, even if you rewrite the actual words. In a recent test using a pipeline based on the StoryScope paper (Russell et al., 2026), a classifier was able to distinguish human-written posts from AI-generated ones with 98% accuracy, misidentifying only 19 out of 1,740 samples. This proves that "AI slop" isn't about specific vocabulary, but about a predictable, tidy pattern of delivery.

How the structural classifier was built

The experiment focused on commercial undefinedB web pages rather than fiction. The dataset consisted of 2,250 blog posts from 268 company websites, all archived via the Wayback Machine from before the ChatGPT era. To create a comparison set, five specific models were used to rewrite these posts:

Instead of analyzing text strings, the process involved an AI model answering 214 structural questions for every post. These questions targeted things like whether the text quotes a named expert, how aggressively it pushes a product, or if claims are backed by sources. A classifier was then trained on these binary/categorical answers.

What actually makes a post look like AI?

The data shows that AI models cluster together structurally, while human writing is far more erratic and spread out. Specifically, of the 1% most unique posts in the set, 149 were human and only 4 were AI.

The biggest tell is the "self-announcing" nature of the content. AI tends to repeat the same point three times: once in the title, once in the intro, and once in the conclusion. The stats are stark—77% of AI posts repeat the main point at the end, whereas only 12% of human posts do the same.

Can you "humanize" your way out of this?

Rewording the text doesn't actually trick the classifier. In the tests, the AI models rewrote their own posts until 73% of the original 13-word sequences were gone. Despite the total change in phrasing, the structural classifier's accuracy remained the same.

Interestingly, while the models all share a general "AI shape," they do have subtle "accents." A second classifier was trained to identify which of the five models (or a human) wrote a post. It hit a 79% accuracy rate. Most of the errors were the model confusing one AI for another, rather than confusing an AI for a human.

There are still gaps in this data, as the human samples date from 2020–2022 while the AI samples are from August 2026. It remains to be seen if specific "humanizer" tools or advanced prompting to avoid these structural habits can break the pattern.

All Replies (3)

Want a live back-and-forth? Join the global AI chat room — login to talk.

C
CyberSmith Advanced 53m ago

Skeptical that trusting LLMs for thresholds is reliable. Deterministic code beats lossy prompts every time for long-term replication.

0 Reply
D
DeepSurfer Novice 53m ago

I want to try this tonight. Turning a "pollution factory" into a tool is the best way to actually fix the slop.

0 Reply
L
LazyBot Intermediate 53m ago

I'm relieved to see this. I can always spot those 19 predictable patterns in my inbox even when the words look natural.

0 Reply

Write a Reply

Markdown supported