Model Collapse: Are New Coding LLMs Training on Old AI Slop?

The "Model Collapse" theory suggests that as AI-generated code floods GitHub and StackOverflow, newer LLMs are essentially eating their own exhaust, leading to a degradation in quality. If a model is trained on data produced by a previous generation of models—especially the mediocre, hallucinated, or "slop" versions of early coding assistants—the resulting output tends to converge toward a mean, losing the edge cases and creative problem-solving that human developers provide.

The Feedback Loop Problem

When we look at the current AI workflow, the volume of synthetic data is exploding. A developer uses an LLM to scaffold a project, pushes that code to a public repo, and then a future training scrape picks up that exact snippet. The danger here isn't just that the code is "bad," but that it's "average." Human code contains intent, architectural trade-offs, and often, the "ugly" fixes that actually work in production. AI slop tends to be syntactically correct but logically hollow.

If a new model trains on a million instances of a slightly wrong implementation of a sorting algorithm because that's what the previous model hallucinated most often, the new model will perceive that error as the "standard" way to write the function.

How Modern Training Mitigates This

To prevent this slide into mediocrity, researchers are moving toward more sophisticated data curation. It's no longer about scraping the entire web; it's about high-signal filtering. Here is how the industry is attempting to avoid the slop:

• Quality Scoring: Using "reward models" to score code snippets. If a piece of code doesn't pass a linter or fails a unit test, it's discarded regardless of how many times it appears in the training set.
• Synthetic Data Generation (The Good Kind): Instead of training on "wild" AI slop, developers are creating high-quality synthetic datasets. This involves using a frontier model (like Claude 3.5 Sonnet) to generate a problem, then using a compiler to verify the solution, and only then adding it to the training set.
• Weighting Human Data: Giving higher priority to "gold standard" repositories—projects with high stars, rigorous PR reviews, and long histories of human maintenance.

Real-World Implications for Prompt Engineering

For those of us focusing on prompt engineering, this means we can't trust the "most common" answer the AI gives us. Since the models are exposed to so much synthetic noise, the "average" response is often the safest, most generic version of the code.

To get around this, I've found that forcing the model to think through a "first-principles" approach or providing a specific, high-quality reference implementation in the prompt helps bypass the generic patterns the model has picked up from the AI-generated noise of the web. The goal is to push the LLM away from the "average" synthetic response and back toward the logic of a senior engineer.