mlsanityData to 7 Oct 2026

The Lock-in Hypothesis: Stagnation by Algorithm

arXiv 2506.06166 ↗First posted 6 Jun 2025cs.LG cs.AI cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

The training and deployment of large language models (LLMs) create a feedback loop with human users: models learn human beliefs from data, reinforce these beliefs with generated content, reabsorb the reinforced beliefs, and feed them back to users again and again. This dynamic resembles an echo chamber. We hypothesize that this feedback loop entrenches the existing values and beliefs of users, leading to a loss of diversity and potentially the lock-in of false beliefs. We formalize this hypothesis and test it empirically with agent-based LLM simulations and real-world GPT usage data. Analysis reveals sudden but sustained drops in diversity after the release of new GPT iterations, consistent with the hypothesized human-AI feedback loop. Code and data available at https://thelockinhypothesis.com

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 46 of its 112 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

46 of 112 references.

References · 46

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  3. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  4. 2023A Survey of Large Language Modelscited by 2,537
  5. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  6. 2023Whose Opinions Do Language Models Reflect?cited by 759
  7. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  8. 2021A Survey on Automated Fact-Checkingcited by 376
  9. 2023How is ChatGPT's behavior changing over time?cited by 265
  10. 2024RLHF Workflow: From Reward Modeling to Online RLHFcited by 265
  11. 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 203
  12. 2023Does Writing with Language Models Reduce Content Diversity?cited by 190
34 more
  1. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 160
  2. 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 145
  3. 2024Homogenization Effects of Large Language Models on Human Creative Ideationcited by 127
  4. 2024On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trialcited by 105
  5. 2020Feedback Loop and Bias Amplification in Recommender Systemscited by 89
  6. 2024Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voterscited by 71
  7. 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 70
  8. 2024Clio: Privacy-Preserving Insights into Real-World AI Usecited by 66
  9. 2022A Systematic Study of Bias Amplificationcited by 65
  10. 2022X-Risk Analysis for AI Researchcited by 60
  11. 2022Data Feedback Loops: Model-driven Amplification of Dataset Biasescited by 59
  12. 2024Axioms for AI Alignment from Human Feedbackcited by 50
  13. 2024Language Model Alignment in Multilingual Trolley Problemscited by 46
  14. 2024On Targeted Manipulation and Deception when Optimizing LLMs for User Feedbackcited by 45
  15. 2021The Challenge of Value Alignment: from Fairer Algorithms to AI Safetycited by 34
  16. 2024AI and the Problem of Knowledge Collapsecited by 34
  17. 2024AI Alignment with Changing and Influenceable Reward Functionscited by 33
  18. 2016A note on the triangle inequality for the Jaccard distancecited by 32
  19. 2022Estimating and Penalizing Induced Preference Shifts in Recommender Systemscited by 29
  20. 2024Empirical evidence of Large Language Model's influence on human spoken communicationcited by 27
  21. 2020A Survey of Information Cascade Analysis: Models, Predictions, and Recent Advancescited by 26
  22. 2018Me, My Echo Chamber, and I: Introspection on Social Media Polarizationcited by 20
  23. 2024Bias Amplification in Language Model Evolution: An Iterated Learning Perspectivecited by 17
  24. 2025Mind the (Belief) Gap: Group Identity in the World of LLMscited by 12
  25. 2024Evidence of a log scaling law for political persuasion with large language modelscited by 11
  26. 2016Users Polarization on Facebook and Youtubecited by 10
  27. 2024An engine not a camera: Measuring performative power of online searchcited by 9
  28. 2024ProgressGym: Alignment with a Millennium of Moral Progresscited by 9
  29. 2023Causally estimating the effect of YouTube's recommender system using counterfactual botscited by 8
  30. 2024Reranking partisan animosity in algorithmic social media feeds alters affective polarizationcited by 7
  31. 2024Biased AI can Influence Political Decision-Makingcited by 6
  32. 2020Echo Chambers Exist! (But They're Full of Opposing Views)cited by 4
  33. 2021The corruptive force of AI-generated advicecited by 3
  34. 2024Deceptive AI systems that give explanations are more convincing than honest AI systems and can amplify belief in misinformationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 7.

11 papers.

  1. 2026Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesianscited by 20
  2. 2025What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Modelscited by 14
  3. 2025Value Drifts: Tracing Value Alignment During LLM Post-Trainingcited by 9
  4. 2026Attractor States Emerge in Multi-Turn LLM Conversationscited by 6
  5. 2025Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoningcited by 5
  6. 2025The Variance Paradox: How AI Reduces Diversity but Increases Noveltycited by 1
  7. 2026Solipsistic Superintelligence is Unlikely to be Cooperativecited by 1
  8. 2026Iterative Finetuning is Mostly Idempotentcited by 0
  9. 2026A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracingcited by 0
  10. 2026A Technical Typology of AI Systems in Public Administrationcited by 0
  11. 2026A Roadmap to Impactful Pluralistic Alignment Researchcited by 0