# The Lock-in Hypothesis: Stagnation by Algorithm

arXiv: https://arxiv.org/abs/2506.06166 · HTML: https://mlsanity.com/paper/2506.06166/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 112 references that are in the map.
- First posted on arXiv 6 Jun 2025.

## Builds on (most cited first, 30 of 46)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [RLHF Workflow: From Reward Modeling to Online RLHF](https://mlsanity.com/paper/2405.07863/) (2024)
- [Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization](https://mlsanity.com/paper/2406.01171/) (2024)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews](https://mlsanity.com/paper/2403.07183/) (2024)
- [Co-Writing with Opinionated Language Models Affects Users' Views](https://mlsanity.com/paper/2302.00560/) (2023)
- [Homogenization Effects of Large Language Models on Human Creative Ideation](https://mlsanity.com/paper/2402.01536/) (2024)
- [On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial](https://mlsanity.com/paper/2403.14380/) (2024)
- [Feedback Loop and Bias Amplification in Recommender Systems](https://mlsanity.com/paper/2007.13019/) (2020)
- [Feedback Loops With Language Models Drive In-Context Reward Hacking](https://mlsanity.com/paper/2402.06627/) (2024)
- [Clio: Privacy-Preserving Insights into Real-World AI Use](https://mlsanity.com/paper/2412.13678/) (2024)
- [Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voters](https://mlsanity.com/paper/2410.24190/) (2024)
- [A Systematic Study of Bias Amplification](https://mlsanity.com/paper/2201.11706/) (2022)
- [X-Risk Analysis for AI Research](https://mlsanity.com/paper/2206.05862/) (2022)
- [Data Feedback Loops: Model-driven Amplification of Dataset Biases](https://mlsanity.com/paper/2209.03942/) (2022)
- [Axioms for AI Alignment from Human Feedback](https://mlsanity.com/paper/2405.14758/) (2024)
- [On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback](https://mlsanity.com/paper/2411.02306/) (2024)
- [Language Model Alignment in Multilingual Trolley Problems](https://mlsanity.com/paper/2407.02273/) (2024)
- [The Challenge of Value Alignment: from Fairer Algorithms to AI Safety](https://mlsanity.com/paper/2101.06060/) (2021)
- [AI and the Problem of Knowledge Collapse](https://mlsanity.com/paper/2404.03502/) (2024)
- [AI Alignment with Changing and Influenceable Reward Functions](https://mlsanity.com/paper/2405.17713/) (2024)
- [A note on the triangle inequality for the Jaccard distance](https://mlsanity.com/paper/1612.02696/) (2016)

## Built on by (most cited first, 11 of 11)

- [Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians](https://mlsanity.com/paper/2602.19141/) (2026)
- [What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Models](https://mlsanity.com/paper/2510.04226/) (2025)
- [Value Drifts: Tracing Value Alignment During LLM Post-Training](https://mlsanity.com/paper/2510.26707/) (2025)
- [Attractor States Emerge in Multi-Turn LLM Conversations](https://mlsanity.com/paper/2606.30571/) (2026)
- [Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoning](https://mlsanity.com/paper/2512.02914/) (2025)
- [The Variance Paradox: How AI Reduces Diversity but Increases Novelty](https://mlsanity.com/paper/2508.19264/) (2025)
- [Solipsistic Superintelligence is Unlikely to be Cooperative](https://mlsanity.com/paper/2606.03237/) (2026)
- [Iterative Finetuning is Mostly Idempotent](https://mlsanity.com/paper/2605.01130/) (2026)
- [A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing](https://mlsanity.com/paper/2606.05330/) (2026)
- [A Technical Typology of AI Systems in Public Administration](https://mlsanity.com/paper/2606.31755/) (2026)
- [A Roadmap to Impactful Pluralistic Alignment Research](https://mlsanity.com/paper/2607.22305/) (2026)

## Abstract (arXiv)

The training and deployment of large language models (LLMs) create a feedback loop with human users: models learn human beliefs from data, reinforce these beliefs with generated content, reabsorb the reinforced beliefs, and feed them back to users again and again. This dynamic resembles an echo chamber. We hypothesize that this feedback loop entrenches the existing values and beliefs of users, leading to a loss of diversity and potentially the lock-in of false beliefs. We formalize this hypothesis and test it empirically with agent-based LLM simulations and real-world GPT usage data. Analysis reveals sudden but sustained drops in diversity after the release of new GPT iterations, consistent with the hypothesized human-AI feedback loop. Code and data available at https://thelockinhypothesis.com