From Tool to Agent: How Worker Needs Reorganize Across the Agentic Roles of Workplace AI
Not on the map. Why?
Abstract (arXiv)
As AI systems gain agency in workplaces, workers increasingly interact with systems that do more than support tasks: they make decisions, allocate work, and shape how workers communicate with others. We interviewed 16 workers in healthcare, finance, and management about their daily interactions with workplace AI systems. Across these domains, AI occupied three agentic roles: assistant, supporting worker decision-making; arbiter, making consequential decisions that workers had to explain to downstream stakeholders; and manager, directing work and displacing social structures through which work had been negotiated and learned. We find that worker needs depended on how decision authority and accountability were configured around the AI system. Explanation, control, accountability, and communication each took different meanings depending on whether AI supported, decided, or directed work. We contribute a role-based account of workplace human-agent interaction and identify accountability-authority asymmetry as a mechanism through which AI deployments can place responsibility on workers for outcomes whose authority has shifted to AI. We discuss implications for designing workplace AI around both the agent and its surrounding agency configuration.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 57 references that are in the map. First posted on arXiv 4 May 2026.
Builds on
22 of 57 references.
References · 22
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
- 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
- 2020Questioning the AI: Informing Design Practices for Explainable AI User Experiencescited by 251
- 2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
- 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
- 2023Levels of AGI for Operationalizing Progress on the Path to AGIcited by 94
- 2025Levels of Autonomy for AI Agentscited by 45
- 2025Fully Autonomous AI Agents Should Not be Developedcited by 43
- 2022Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Supportcited by 37
- 2021Designing AI for Trust and Collaboration in Time-Constrained Medical Decisions: A Sociotechnical Lenscited by 35
- 2023Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Servicescited by 29
- 2023How Knowledge Workers Think Generative AI Will (Not) Transform Their Industriescited by 29
10 more
- 2025VeriPlan: Integrating Formal Verification and LLMs into End-User Planningcited by 19
- 2026From Future of Work to Future of Workers: Addressing Asymptomatic AI Harms for Dignified Human-AI Interactioncited by 6
- 2024REX: Designing User-centered Repair and Explanations to Address Robot Failurescited by 5
- 2025Generative AI Uses and Risks for Knowledge Workers in a Science Organizationcited by 5
- 2025When Your Boss Is an AI Bot: Exploring Opportunities and Risks of Manager Clone Agents in the Future Workplacecited by 3
- 2025Regulating Algorithmic Management: A Multi-Stakeholder Study of Challenges in Aligning Software and the Law for Workplace Schedulingcited by 2
- 2026The Values of Value in AI Adoption: Rethinking Efficiency in UX Designers' Workplacescited by 2
- 2022Consent as a Foundation for Responsible Autonomycited by 1
- 2024Investigating Why Clinicians Deviate from Standards of Care: Liberating Patients from Mechanical Ventilation in the ICUcited by 1
- 2025Fulfillment of the Work Games: Warehouse Workers' Experiences with Algorithmic Managementcited by 1
Built on by
Not cited yet.