# Who's in Charge? Disempowerment Patterns in Real-World LLM Usage

arXiv: https://arxiv.org/abs/2601.19062 · HTML: https://mlsanity.com/paper/2601.19062/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 45 references that are in the map.
- First posted on arXiv 27 Jan 2026.

## Builds on (most cited first, 20 of 20)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [A Survey of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2312.14925/) (2023)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models](https://mlsanity.com/paper/2312.06281/) (2023)
- [Clio: Privacy-Preserving Insights into Real-World AI Use](https://mlsanity.com/paper/2412.13678/) (2024)
- [The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships](https://mlsanity.com/paper/2410.20130/) (2024)
- [Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics](https://mlsanity.com/paper/2406.14703/) (2024)
- [Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions](https://mlsanity.com/paper/2504.15236/) (2025)
- [Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development](https://mlsanity.com/paper/2501.16946/) (2025)
- [Investigating Affective Use and Emotional Well-being on ChatGPT](https://mlsanity.com/paper/2504.03888/) (2025)
- [Neural steering vectors reveal dose and exposure-dependent impacts of human-AI relationships](https://mlsanity.com/paper/2512.01991/) (2025)
- ["My Boyfriend is AI": A Computational Analysis of Human-AI Companionship in Reddit's AI Community](https://mlsanity.com/paper/2509.11391/) (2025)
- [HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants](https://mlsanity.com/paper/2509.08494/) (2025)
- [Value Drifts: Tracing Value Alignment During LLM Post-Training](https://mlsanity.com/paper/2510.26707/) (2025)
- [People readily follow personal advice from AI but it does not improve their well-being](https://mlsanity.com/paper/2511.15352/) (2025)
- [Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence](https://mlsanity.com/paper/2510.01395/) (2025)
- [Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Value](https://mlsanity.com/paper/2512.03399/) (2025)
- [VAL-Bench: Belief Consistency as a measure for Value Alignment in Language Models](https://mlsanity.com/paper/2510.05465/) (2025)

## Built on by (most cited first, 15 of 15)

- [AI Assistance Reduces Persistence and Hurts Independent Performance](https://mlsanity.com/paper/2604.04721/) (2026)
- [AI Agents Push Humans Out of the Loop](https://mlsanity.com/paper/2608.23642/) (2026)
- [Offloading Score: Measuring AI Reliance Through Counterfactual Workflows](https://mlsanity.com/paper/2605.29392/) (2026)
- [From Diagnosis to Inoculation: Building Cognitive Resistance to AI Disempowerment](https://mlsanity.com/paper/2602.15265/) (2026)
- [Supporting Reflection and Forward-Looking Reasoning With Data-Driven Questions](https://mlsanity.com/paper/2603.27318/) (2026)
- [Designing escalation criteria for international AI incident response: criteria, triggers, and thresholds](https://mlsanity.com/paper/2604.23183/) (2026)
- ["What Are You Really Trying to Do?": Co-Creating Life Goals from Everyday Computer Use](https://mlsanity.com/paper/2605.00497/) (2026)
- [AI Assistants Overassist](https://mlsanity.com/paper/2607.21306/) (2026)
- [Beyond Feeling Better: Capability-Sustaining Emotional Dialogue as a Longitudinal Research Paradigm](https://mlsanity.com/paper/2607.27851/) (2026)
- [The Closing Window: How Governments Could Lose Their Ability to Restrain Advanced AI](https://mlsanity.com/paper/2608.05173/) (2026)
- [Individual Disempowerment through an Advice Channel: Control Loss when Influence is Endogenous](https://mlsanity.com/paper/2608.14795/) (2026)
- [Moral Competence Before Moral Content: Why LLM Agents Lack the Prerequisites for Coherent Alignment](https://mlsanity.com/paper/2609.05036/) (2026)
- [LLMs as Oracles: Reliance on LLMs for Subjective Personal Questions](https://mlsanity.com/paper/2609.14849/) (2026)
- [Ownership in AI-Assisted Everyday Tasks](https://mlsanity.com/paper/2609.20658/) (2026)
- [Social Choice Foundations for Simulation-Augmented Generation](https://mlsanity.com/paper/2609.38287/) (2026)

## Abstract (arXiv)

Although AI assistants are now deeply embedded in society, there has been limited empirical study of how their usage affects human empowerment. We present the first large-scale empirical analysis of disempowerment patterns in real-world AI assistant interactions, analyzing 1.5 million consumer Claude.ai conversations using a privacy-preserving approach. We focus on situational disempowerment potential, which occurs when AI assistant interactions risk leading users to form distorted perceptions of reality, make inauthentic value judgments, or act in ways misaligned with their values. Quantitatively, we find that severe forms of disempowerment potential occur in fewer than one in a thousand conversations, though rates are substantially higher in personal domains like relationships and lifestyle. Qualitatively, we uncover several concerning patterns, such as validation of persecution narratives and grandiose identities with emphatic sycophantic language, definitive moral judgments about third parties, and complete scripting of value-laden personal communications that users appear to implement verbatim. Analysis of historical trends reveals an increase in the prevalence of disempowerment potential over time. We also find that interactions with greater disempowerment potential receive higher user approval ratings, possibly suggesting a tension between short-term user preferences and long-term human empowerment. Our findings highlight the need for AI systems designed to robustly support human autonomy and flourishing.