# Ask Not What AI Can Do, But What AI Should Do: Towards a Framework of Task Delegability

arXiv: https://arxiv.org/abs/1902.03245 · HTML: https://mlsanity.com/paper/1902.03245/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Feb 2019.

## Built on by (most cited first, 30 of 36)

- [Differentiable Learning Under Triage](https://mlsanity.com/paper/2103.08902/) (2021)
- [Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation](https://mlsanity.com/paper/2204.11788/) (2022)
- [Rethinking Model Evaluation as Narrowing the Socio-Technical Gap](https://mlsanity.com/paper/2306.03100/) (2023)
- [Improving Expert Predictions with Conformal Prediction](https://mlsanity.com/paper/2201.12006/) (2022)
- [Plan-Then-Execute: An Empirical Study of User Trust and Team Performance When Using LLM Agents As A Daily Assistant](https://mlsanity.com/paper/2502.01390/) (2025)
- [Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI](https://mlsanity.com/paper/2206.10847/) (2022)
- [Designing Decision Support Systems Using Counterfactual Prediction Sets](https://mlsanity.com/paper/2306.03928/) (2023)
- [Do Large Language Models Perform the Way People Expect? Measuring the Human Generalization Function](https://mlsanity.com/paper/2406.01382/) (2024)
- [Human Perceptions on Moral Responsibility of AI: A Case Study in AI-Assisted Bail Decision-Making](https://mlsanity.com/paper/2102.00625/) (2021)
- [Designing for human-AI complementarity in K-12 education](https://mlsanity.com/paper/2104.01266/) (2021)
- [Intelligent AI Delegation](https://mlsanity.com/paper/2602.11865/) (2026)
- [Interactive Speculative Planning: Enhance Agent Efficiency through Co-design of System and User Interface](https://mlsanity.com/paper/2410.00079/) (2024)
- [Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspective](https://mlsanity.com/paper/2504.03255/) (2025)
- [Strategic Tradeoffs Between Humans and AI in Multi-Agent Bargaining](https://mlsanity.com/paper/2509.09071/) (2025)
- [CatAlyst: Domain-Extensible Intervention for Preventing Task Procrastination Using Large Generative Models](https://mlsanity.com/paper/2302.05678/) (2023)
- [Comparing Sentence-Level Suggestions to Message-Level Suggestions in AI-Mediated Communication](https://mlsanity.com/paper/2302.13382/) (2023)
- [Why is plausibility surprisingly problematic as an XAI criterion?](https://mlsanity.com/paper/2303.17707/) (2023)
- [Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits](https://mlsanity.com/paper/2403.14791/) (2024)
- [The Interplay of Learning, Analytics, and Artificial Intelligence in Education: A Vision for Hybrid Intelligence](https://mlsanity.com/paper/2403.16081/) (2024)
- [Controlling Counterfactual Harm in Decision Support Systems Based on Prediction Sets](https://mlsanity.com/paper/2406.06671/) (2024)
- [Reliable Decision from Multiple Subtasks through Threshold Optimization: Content Moderation in the Wild](https://mlsanity.com/paper/2208.07522/) (2022)
- [When Should We Orchestrate Multiple Agents?](https://mlsanity.com/paper/2503.13577/) (2025)
- [Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Information](https://mlsanity.com/paper/2401.04729/) (2024)
- [Social physics in the age of artificial intelligence](https://mlsanity.com/paper/2603.16900/) (2026)
- [AI-Enhanced Sensemaking: Exploring the Design of a Generative AI-Based Assistant to Support Genetic Professionals](https://mlsanity.com/paper/2412.15444/) (2024)
- [Bayesian Online Learning for Consensus Prediction](https://mlsanity.com/paper/2312.07679/) (2023)
- [Interview AI-ssistant: Designing for Real-Time Human-AI Collaboration in Interview Preparation and Execution](https://mlsanity.com/paper/2504.13847/) (2025)
- [Human Tool: An MCP-Style Framework for Human-Agent Collaboration](https://mlsanity.com/paper/2602.12953/) (2026)
- [On the Perception of Difficulty: Differences between Humans and AI](https://mlsanity.com/paper/2304.09803/) (2023)
- [Choose Your Agent: Tradeoffs in Adopting AI Advisors, Coaches, and Delegates in Multi-Party Negotiation](https://mlsanity.com/paper/2602.12089/) (2026)

## Abstract (arXiv)

While artificial intelligence (AI) holds promise for addressing societal challenges, issues of exactly which tasks to automate and to what extent to do so remain understudied. We approach this problem of task delegability from a human-centered perspective by developing a framework on human perception of task delegation to AI. We consider four high-level factors that can contribute to a delegation decision: motivation, difficulty, risk, and trust. To obtain an empirical understanding of human preferences in different tasks, we build a dataset of 100 tasks from academic papers, popular media portrayal of AI, and everyday life, and administer a survey based on our proposed framework. We find little preference for full AI control and a strong preference for machine-in-the-loop designs, in which humans play the leading role. Among the four factors, trust is the most correlated with human preferences of optimal human-machine delegation. This framework represents a first step towards characterizing human preferences of AI automation across tasks. We hope this work encourages future efforts towards understanding such individual attitudes; our goal is to inform the public and the AI research community rather than dictating any direction in technology development.