# HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants

arXiv: https://arxiv.org/abs/2509.08494 · HTML: https://mlsanity.com/paper/2509.08494/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 122 references that are in the map.
- First posted on arXiv 10 Sep 2025.

## Builds on (most cited first, 30 of 44)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2307.15217/) (2023)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024)
- [FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets](https://mlsanity.com/paper/2307.10928/) (2023)
- [Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations](https://mlsanity.com/paper/2411.00640/) (2024)
- [Asking Clarifying Questions in Open-Domain Information-Seeking Conversations](https://mlsanity.com/paper/1907.06554/) (2019)
- [On the Risk of Misinformation Pollution with Large Language Models](https://mlsanity.com/paper/2305.13661/) (2023)
- [Survey on Evaluation Methods for Dialogue Systems](https://mlsanity.com/paper/1905.04071/) (2019)
- [Collective Constitutional AI: Aligning a Language Model with Public Input](https://mlsanity.com/paper/2406.07814/) (2024)
- [Optimal Policies Tend to Seek Power](https://mlsanity.com/paper/1912.01683/) (2019)
- [Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information](https://mlsanity.com/paper/1805.04655/) (2018)
- [The Ethics of Advanced AI Assistants](https://mlsanity.com/paper/2404.16244/) (2024)
- [Bridging the Gulf of Envisioning: Cognitive Design Challenges in LLM Interfaces](https://mlsanity.com/paper/2309.14459/) (2023)
- [Hidden Persuaders: LLMs' Political Leaning and Their Influence on Voters](https://mlsanity.com/paper/2410.24190/) (2024)
- [CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation](https://mlsanity.com/paper/2311.18702/) (2023)
- [The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships](https://mlsanity.com/paper/2410.20130/) (2024)
- [Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise](https://mlsanity.com/paper/2410.03017/) (2024)
- [Thousands of AI Authors on the Future of AI](https://mlsanity.com/paper/2401.02843/) (2024)
- [Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development](https://mlsanity.com/paper/2501.16946/) (2025)
- [Toward an Evaluation Science for Generative AI Systems](https://mlsanity.com/paper/2503.05336/) (2025)
- [Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation](https://mlsanity.com/paper/2402.12649/) (2024)
- [Understanding accountability in algorithmic supply chains](https://mlsanity.com/paper/2304.14749/) (2023)
- [Generative AI for Synthetic Data Generation: Methods, Challenges and the Future](https://mlsanity.com/paper/2403.04190/) (2024)

## Built on by (most cited first, 11 of 11)

- [Characterizing Delusional Spirals through Human-LLM Chat Logs](https://mlsanity.com/paper/2603.16567/) (2026)
- [Digital Companionship: Overlapping Uses of AI Companions and AI Assistants](https://mlsanity.com/paper/2510.15905/) (2025)
- [AI and Human Oversight: A Risk-Based Framework for Alignment](https://mlsanity.com/paper/2510.09090/) (2025)
- [Mental Models of Autonomy and Sentience Shape Reactions to AI](https://mlsanity.com/paper/2512.09085/) (2025)
- [ForesightSafety Bench: A Frontier Risk Evaluation and Governance Framework towards Safe AI](https://mlsanity.com/paper/2602.14135/) (2026)
- [Effects of Generative AI Errors on User Reliance Across Task Difficulty](https://mlsanity.com/paper/2604.04319/) (2026)
- [What Did They Mean? How LLMs Resolve Ambiguous Social Situations across Perspectives and Roles](https://mlsanity.com/paper/2604.23942/) (2026)
- [Pro-AI Bias in Large Language Models](https://mlsanity.com/paper/2601.13749/) (2026)
- [Althea: The Fact-Checking--Metalearning Tradeoff in AI-Assisted Verification](https://mlsanity.com/paper/2602.11161/) (2025)
- [Individual Disempowerment through an Advice Channel: Control Loss when Influence is Endogenous](https://mlsanity.com/paper/2608.14795/) (2026)
- [Method, Mind, and Morality: How People Make Sense of Artificial Intelligence](https://mlsanity.com/paper/2608.24748/) (2026)

## Abstract (arXiv)

As humans delegate more tasks and decisions to artificial intelligence (AI), we risk losing control of our individual and collective futures. Relatively simple algorithmic systems already steer human decision-making, such as social media feed algorithms that lead people to unintentionally and absent-mindedly scroll through engagement-optimized content. In this paper, we develop the idea of human agency by integrating philosophical and scientific theories of agency with AI-assisted evaluation methods: using large language models (LLMs) to simulate and validate user queries and to evaluate AI responses. We develop HumanAgencyBench (HAB), a scalable and adaptive diagnostic tool for six behaviors related to human agency. HAB measures the tendency of an AI assistant to Ask Clarifying Questions, Avoid Value Manipulation, Correct Misinformation, Defer Important Decisions, Encourage Learning, and Maintain Social Boundaries. We find low-to-moderate agency support in contemporary LLM-based assistants, with substantial variation across system developers and behaviors. For example, while Anthropic LLMs most support human agency overall, they are the least supportive LLMs in terms of Avoid Value Manipulation. These behaviors do not appear to consistently result from increasing LLM capabilities or instruction-following (e.g., RLHF); we encourage further study of these behaviors so that developers and users can better understand the complexities of modern human-AI interaction.