Giving AI More Data Can Actually Make It Less Reliable

Bigger datasets used to mean better AI. A 2024 Nature study shows why more data can now make models less reliable instead of more capable.

Feeding a language model more training data does not automatically make it smarter. Past a certain point, adding more data, especially uncurated or AI-generated data, can degrade a model's outputs rather than improve them. A 2024 study published in Nature by researchers at Oxford and Cambridge documented a specific failure mode behind this, called model collapse, and it has reshaped how labs think about scaling.

The Scaling Law Everyone Assumed Would Hold Forever

The idea that bigger datasets reliably produce better models comes from scaling laws, relationships researchers established showing that model performance improves predictably as parameters and training tokens increase. That pattern held from GPT-3 onward and became the industry's default growth strategy: more data, more compute, better results.

The assumption baked into that strategy was that additional data resembled the data before it, meaning real, diverse, human-generated text and images. Once a model has already absorbed a broad, high-quality sample of a domain, extra data that is noisy, redundant, or synthetic contributes confusion rather than new signal, which is why the returns on pure scale have visibly slowed since 2024.

What Model Collapse Actually Is

Model collapse is a degenerative process where an AI system trained on data generated by earlier AI systems gradually loses touch with the true, original distribution of human-produced content. Researcher Ilia Shumailov and colleagues at Oxford, Cambridge, and Imperial College London formalized the mechanism in a 2024 Nature paper, testing it across large language models, variational autoencoders, and Gaussian mixture models.

Their experiments showed collapse happens in two stages. Early collapse loses rare, unusual examples first, the tail end of the data distribution, because a model is statistically less likely to reproduce something it saw only once. Late collapse is worse: the outputs converge toward a narrow, repetitive average, and by later generations, unrelated gibberish can appear, since the fine distinctions that separated correct answers from wrong ones have been averaged away.

The team ran this on a small language model fine-tuned on Wikipedia text, then fed each new generation's output back in as the next generation's training data. Perplexity, a measure of how well a model predicts real text, climbed by 20 to 28 points across generations trained purely on synthetic output, a clear numeric signature of the model drifting away from coherent, accurate language.

Why This Matters More in 2026 Than It Did in 2023

The web itself has changed in a way that feeds this problem directly. A widely cited estimate found that by April 2025, roughly 74 percent of newly created webpages already contained some AI-generated text, which means models trained on fresh web scrapes are increasingly training on the outputs of earlier models rather than on original human writing.

That creates a feedback loop distinct from ordinary AI training data shortages. It is not that the internet is running out of text. It is that a growing share of new text was written by a machine, and training on it recursively without careful filtering is what triggers the collapse Shumailov's team measured in a laboratory setting.

Reliability Problems That Look Like Something Else

Model collapse is one mechanism, but it is not the only way more data can hurt a model's reliability rather than help it. A model trained on stale or outdated data drifts from present-day facts in ways that are invisible unless someone checks, since fluent phrasing gives no clue that the underlying information is five years old.

Separately, indiscriminately adding low-quality or duplicated data dilutes the signal a model needs to generalize well, a distinct problem from collapse but one that produces a similar symptom. The result reads as fluent and authoritative while being subtly incorrect in ways that need real domain expertise to catch, which is part of why a model can be confidently wrong about a fact it never had solid grounds to state.

What Labs Are Doing Instead of Just Scaling Up

The practical response has been a shift from data volume to data curation. Research through late 2025 and into 2026 has consistently shown that rigorous deduplication, filtering, and domain-specific annotation deliver more performance per training dollar than scaling raw dataset size, and that gap has widened as easily available high-quality text has become scarcer.

Later research, including work by Gerstgrasser and colleagues, found that collapse can be slowed or avoided when labs keep accumulating genuine human-generated data alongside synthetic data rather than replacing it, though this does not fully solve the underlying problem. Detecting and filtering machine-generated text before it enters a training set has become an active research area for exactly this reason, since prevention is proving more tractable than reversing collapse after it happens.

Why a Model Can Sound Certain and Still Be Wrong

None of this is helped by the fact that these systems are not built to flag their own uncertainty by default. A model trained on degraded or thinning data still produces the same confident sentence structure it always has, because fluency and accuracy are generated by different parts of the process. The tone of an answer says nothing about how well-supported it actually was by real training examples, which is a separate problem from data volume but compounds it. It is one reason researchers studying AI hallucination describe confidence and correctness as two things that can drift apart without any external sign.

What This Means for Anyone Using AI Tools Right Now

None of this means today's models are collapsing. Frontier labs actively monitor for it and mix in curated, verified data specifically to avoid it. What it does mean is that the assumption "more data equals a better model" no longer holds without qualification, and a model's confident tone is not evidence that a claim rests on solid training data. The next leap in AI capability is more likely to come from smarter data curation and architecture than from simply feeding models a larger pile of the internet.

Frequently Asked Questions

What is AI model collapse?

It is a degenerative process where a model trained repeatedly on AI-generated data gradually loses rare, accurate patterns and drifts toward bland, sometimes nonsensical, average outputs.

Does adding more training data always improve an AI model?

No. Past a certain point, additional data that is noisy, redundant, or AI-generated can reduce reliability instead of improving it.

Can model collapse be prevented?

Research suggests it can be slowed by mixing genuine human-generated data alongside synthetic data and by filtering out machine-generated text before training, though it has not been fully solved.

Is the internet running out of data for AI training?

High-quality, original human text is becoming scarcer relative to demand, and researchers have projected a shortage before 2026 if current data-hungry training trends continue.

Should I trust an AI answer just because it sounds confident?

No. A model's fluent, confident tone reflects its language patterns, not whether the underlying training data actually supported that specific claim.