# Artificial Intelligent Disobedience: Rethinking the Agency of Our Artificial Teammates

arXiv: https://arxiv.org/abs/2506.22276 · HTML: https://mlsanity.com/paper/2506.22276/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 74 references that are in the map.
- First posted on arXiv 27 Jun 2025.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Data Augmentation Generative Adversarial Networks](https://mlsanity.com/paper/1711.04340/) (2017)
- [Core Challenges of Social Robot Navigation: A Survey](https://mlsanity.com/paper/2103.05668/) (2021)
- [Principles and Guidelines for Evaluating Social Robot Navigation Algorithms](https://mlsanity.com/paper/2306.16740/) (2023)
- [Conflict Avoidance in Social Navigation -- a Survey](https://mlsanity.com/paper/2106.12113/) (2021)
- [The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users](https://mlsanity.com/paper/1301.7385/) (2013)
- ["I am the follower, also the boss": Exploring Different Levels of Autonomy and Machine Forms of Guiding Robots for the Visually Impaired](https://mlsanity.com/paper/2302.03481/) (2023)
- [Exploring the Cost of Interruptions in Human-Robot Teaming](https://mlsanity.com/paper/2311.00785/) (2023)
- [ODGR: Online Dynamic Goal Recognition](https://mlsanity.com/paper/2407.16220/) (2024)
- [Birds of a Different Feather Flock Together: Exploring Opportunities and Challenges in Animal-Human-Machine Teaming](https://mlsanity.com/paper/2504.13973/) (2025)
- [General Dynamic Goal Recognition using Goal-Conditioned and Meta Reinforcement Learning](https://mlsanity.com/paper/2505.09737/) (2025)

## Built on by (most cited first, 4 of 4)

- [GRAIL: Goal Recognition Alignment through Imitation Learning](https://mlsanity.com/paper/2602.14252/) (2026)
- [The Intelligent Disobedience Game: Formulating Disobedience in Stackelberg Games and Markov Decision Processes](https://mlsanity.com/paper/2603.20994/) (2026)
- [Beyond Compliance: How AI Could Help Creative Writers by Refusing Them](https://mlsanity.com/paper/2605.16272/) (2026)
- [Towards Responsibly Non-Compliant Machines](https://mlsanity.com/paper/2606.12147/) (2026)

## Abstract (arXiv)

Artificial intelligence has made remarkable strides in recent years, achieving superhuman performance across a wide range of tasks. Yet despite these advances, most cooperative AI systems remain rigidly obedient, designed to follow human instructions without question and conform to user expectations, even when doing so may be counterproductive or unsafe. This paper argues for expanding the agency of AI teammates to include textitintelligent disobedience, empowering them to make meaningful and autonomous contributions within human-AI teams. It introduces a scale of AI agency levels and uses representative examples to highlight the importance and growing necessity of treating AI autonomy as an independent research focus in cooperative settings. The paper then explores how intelligent disobedience manifests across different autonomy levels and concludes by proposing initial boundaries and considerations for studying disobedience as a core capability of artificial agents.