mlsanityData to 14 Oct 2026

AI and the Problem of Knowledge Collapse

arXiv 2404.03502 ↗First posted 4 Apr 2024cs.AI cs.CYcited by 34

Not on the map. Why?

Abstract (arXiv)

While artificial intelligence has the potential to process vast amounts of data, generate new insights, and unlock greater productivity, its widespread adoption may entail unforeseen consequences. We identify conditions under which AI, by reducing the cost of access to certain modes of knowledge, can paradoxically harm public understanding. While large language models are trained on vast amounts of diverse data, they naturally generate output towards the 'center' of the distribution. This is generally useful, but widespread reliance on recursive AI systems could lead to a process we define as "knowledge collapse", and argue this could harm innovation and the richness of human understanding and culture. However, unlike AI models that cannot choose what data they are trained on, humans may strategically seek out diverse forms of knowledge if they perceive them to be worthwhile. To investigate this, we provide a simple model in which a community of learners or innovators choose to use traditional methods or to rely on a discounted AI-assisted process and identify conditions under which knowledge collapse occurs. In our default model, a 20% discount on AI-generated content generates public beliefs 2.3 times further from the truth than when there is no discount. An empirical approach to measuring the distribution of LLM outputs is provided in theoretical terms and illustrated through a specific example comparing the diversity of outputs across different models and prompting styles. Finally, based on the results, we consider further research directions to counteract such outcomes.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 28 of its 107 references that are in the map. First posted on arXiv 4 Apr 2024.

Builds on

28 of 107 references.

References · 28

  1. 2020Scaling Laws for Neural Language Modelscited by 6,586
  2. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  4. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  5. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  6. 2016NIPS 2016 Tutorial: Generative Adversarial Networkscited by 285
  7. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  8. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 232
  9. 2022A Contrastive Framework for Neural Text Generationcited by 225
  10. 2017Generalization and Equilibrium in Generative Adversarial Nets (GANs)cited by 166
  11. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 130
  12. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
16 more
  1. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 103
  2. 2023The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Textcited by 100
  3. 2023Combating Misinformation in the Age of LLMs: Opportunities and Challengescited by 95
  4. 2022On Decoding Strategies for Neural Text Generatorscited by 90
  5. 2019Degenerate Feedback Loops in Recommender Systemscited by 67
  6. 2021User-centered Evaluation of Popularity Bias in Recommender Systemscited by 38
  7. 2022CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender Systemcited by 38
  8. 2023Nepotistically Trained Generative-AI Models Collapsecited by 27
  9. 2024Under the Surface: Tracking the Artifactuality of LLM-Generated Datacited by 24
  10. 2021The Effect of People Recommenders on Echo Chambers and Polarizationcited by 20
  11. 2024Graph-Based Retriever Captures the Long Tail of Biomedical Knowledgecited by 20
  12. 2022Data Debugging with Shapley Importance over End-to-End Machine Learning Pipelinescited by 17
  13. 2024Generative Echo Chamber? Effects of LLM-Powered Search Systems on Diverse Information Seekingcited by 13
  14. 2022Quantifying and Mitigating Popularity Bias in Conversational Recommender Systemscited by 11
  15. 2023In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Searchcited by 9
  16. 2023Knowledge Base Completion for Long-Tail Entitiescited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 15; 2026: 13.

34 papers.

Sort:

By area:

  1. 2026The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systemscited by 30
  2. 2024Do Large Language Models Have an English Accent? Evaluating and Improving the Naturalness of Multilingual LLMscited by 19
  3. 2024The Impact of Large Language Models in Academia: from Writing to Speakingcited by 18
  4. 2024Can We Use Large Language Models to Fill Relevance Judgment Holes?cited by 17
  5. 2024When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settingscited by 17
  6. 2025What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Modelscited by 14
  7. 2025LLM as a Broken Telephone: Iterative Generation Distorts Informationcited by 12
  8. 2025The Lock-in Hypothesis: Stagnation by Algorithmcited by 11
  9. 2024ProgressGym: Alignment with a Millennium of Moral Progresscited by 9
  10. 2025Variance reduction in output from generative AIcited by 6
  11. 2025What happens when generative AI models train recursively on each others' outputs?cited by 6
  12. 2025CrowdLLM: Building LLM-Based Digital Populations Augmented with Generative Modelscited by 5
22 more
  1. 2025Causes and Consequences of Representational Similarity in Machine Learning Modelscited by 3
  2. 2026Markovian Generation Chains in Large Language Modelscited by 3
  3. 2025Convergence Dynamics and Stabilization Strategies of Co-Evolving Generative Modelscited by 2
  4. 2025Epistemic diversity across language models mitigates knowledge collapsecited by 2
  5. 2026The Homogenization Problem in LLMs: Towards Meaningful Diversity in AI Safetycited by 2
  6. 2025AI in the Writing Process: How Purposeful AI Support Fosters Student Writingcited by 1
  7. 2025Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail under Recursive Synthetic Trainingcited by 1
  8. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  9. 2025Big AI is accelerating the metacrisis: What can we do?cited by 1
  10. 2026AI-Mediated Communication Can Steer Collective Opinioncited by 1
  11. 2026Argument Collapse: LLMs Flatten Long-Form Public Debatecited by 1
  12. 2024At the edge of a generative cultural precipicecited by 0
  13. 2025Training for Obsolescence? The AI-Driven Education Trapcited by 0
  14. 2025Future of AI Models: A Computational perspective on Model collapsecited by 0
  15. 2026"Oops! ChatGPT is Temporarily Unavailable!": A Diary Study on Knowledge Workers' Experiences of LLM Withdrawalcited by 0
  16. 2026Using AI in engineering education: a balancing act, driven by clear purposecited by 0
  17. 2026Linguistic Monoculture in LLM-Assisted Language Usecited by 0
  18. 2026When AI Writes, Who Gets Cited? Evidence of Citation Monoculture Across Language Modelscited by 0
  19. 2026RAG Collapse: LLM Responses Collapse When Retrieved Documents Are Self-Authoredcited by 0
  20. 2026Supporting and Performing Culture from the Insidecited by 0
  21. 2026Narrowing the Horizon: Quantifying Topic Saliency Shifts in Generative Monoculturecited by 0
  22. 2026The Uneven Decline of Collective Knowledge Production: Evidence from Stack Overflow After Generative AIcited by 0