mlsanityData to 7 Oct 2026

Fully Autonomous AI Agents Should Not be Developed

arXiv 2502.02649 ↗First posted 4 Feb 2025cs.AIcited by 43

Not on the map. Why?

Cited by 22 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper argues that fully autonomous AI agents should not be developed. In support of this position, we build from prior scientific literature and current product marketing to delineate different AI agent levels and detail the ethical values at play in each, documenting trade-offs in potential benefits and risks. Our analysis reveals that risks to people increase with the autonomy of a system: The more control a user cedes to an AI agent, the more risks to people arise. Particularly concerning are safety risks, which affect human life and impact further values.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 33 of its 201 references that are in the map. First posted on arXiv 4 Feb 2025.

Builds on

33 of 201 references.

References · 33

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  3. 2016Concrete Problems in AI Safetycited by 1,582
  4. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,210
  5. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  6. 2023Large Language Modelscited by 774
  7. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  8. 2018Gender Bias in Coreference Resolutioncited by 438
  9. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
  10. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  11. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 283
  12. 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 235
21 more
  1. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learningcited by 210
  2. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 186
  3. 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 153
  4. 2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
  5. 2024WebCanvas: Benchmarking Web Agents in Online Environmentscited by 119
  6. 2023Levels of AGI for Operationalizing Progress on the Path to AGIcited by 94
  7. 2024AirGapAgent: Protecting Privacy-Conscious Conversational Agentscited by 90
  8. 2024Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistancecited by 84
  9. 2024AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agentscited by 83
  10. 2024The Ethics of ChatGPT in Medicine and Healthcare: A Systematic Review on Large Language Models (LLMs)cited by 45
  11. 2024Security of AI Agentscited by 24
  12. 2022AI Autonomy : Self-Initiated Open-World Continual Learning and Adaptationcited by 23
  13. 2022Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Trafficcited by 15
  14. 2024AI-Powered Autonomous Weapons Risk Geopolitical Instability and Threaten AI Researchcited by 13
  15. 2024Quantifying Misalignment Between Agents: Towards a Sociotechnical Understanding of Alignmentcited by 8
  16. 2014Non-Deterministic Policies in Markovian Decision Processescited by 7
  17. 2024Good Parenting is all you need -- Multi-agentic LLM Hallucination Mitigationcited by 7
  18. 2024Catalyzing Equity in STEM Teams: Harnessing Generative AI for Inclusion and Diversitycited by 6
  19. 2025Observe, Ask, Intervene: Designing AI Agents for More Inclusive Meetingscited by 4
  20. 2020On the Relationship between Software Complexity and Securitycited by 2
  21. 2025AI in Support of Diversity and Inclusioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 26; 2026: 17.

43 papers.

Sort:

By area:

  1. 2025Levels of Autonomy for AI Agentscited by 45
  2. 2025Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforcecited by 33
  3. 2025Characterizing AI Agents for Alignment and Governancecited by 30
  4. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Surveycited by 28
  5. 2025Security Concerns for Large Language Models: A Surveycited by 26
  6. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  7. 2025Agentic AI Needs a Systems Theorycited by 19
  8. 2025Generating Symbolic World Models via Test-time Scaling of Large Language Modelscited by 12
  9. 2025Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspectivecited by 12
  10. 2025A Call for Collaborative Intelligence: Why Human-Agent Systems Should Precede AI Autonomycited by 9
  11. 2025Build Agent Advocates, Not Platform Agentscited by 8
  12. 2025Thoughts without Thinking: Reconsidering the Explanatory Value of Chain-of-Thought Reasoning in LLMs through Agentic Pipelinescited by 7
31 more
  1. 2026How are AI agents used? Evidence from 177,000 MCP toolscited by 7
  2. 2025From surveillance to signalling: escalation channels as environmental controls for agentic AIcited by 6
  3. 2026Modeling Distinct Human Interaction in Web Agentscited by 5
  4. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5
  5. 2025When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaborationcited by 4
  6. 2025Personalizing Agent Privacy Decisions via Logical Entailmentcited by 4
  7. 2025AI Must not be Fully Autonomouscited by 3
  8. 2025Completion neq Collaboration: Scaling Collaborative Effort with Agentscited by 3
  9. 2026Overseeing Agents Without Constant Oversight: Challenges and Opportunitiescited by 3
  10. 2026SafeMCP: Proactive Power Regulation for LLM Agent Defense via Environment-Grounded Look-Ahead Reasoningcited by 3
  11. 2026AI Agents Push Humans Out of the Loopcited by 3
  12. 2025Counterfactual-based Agent Influence Ranker for Agentic AI Workflowscited by 2
  13. 2025Reliable agent engineering should integrate machine-compatible organizational principlescited by 2
  14. 2026Autonomy and Agency in Agentic AI: Architectural Tactics for Regulated Contextscited by 2
  15. 2025I'm Sorry Dave: How the old world of personnel security can inform the new world of AI insider riskcited by 1
  16. 2025OpenAI's GPT-OSS-20B Model and Safety Alignment Issues in a Low-Resource Languagecited by 1
  17. 2025A Criminology of Machinescited by 1
  18. 2025Semantic Deception: When Reasoning Models Can't Compute an Additioncited by 1
  19. 2026Mitigating loss of control in advanced AI systems through instrumental goal trajectoriescited by 1
  20. 2025Agentic AI Optimisation (AAIO): what it is, how it works, why it matters, and how to deal with itcited by 0
  21. 2025AI Agents and Hard Choicescited by 0
  22. 2025Can AI be Accountable?cited by 0
  23. 2026From Tool to Agent: How Worker Needs Reorganize Across the Agentic Roles of Workplace AIcited by 0
  24. 2026Engineering Robustness into Personal Agents with the AI Workflow Storecited by 0
  25. 2026The Agentic Web Requires New Normative Infrastructurecited by 0
  26. 2026SAFETY SENTRY: Context-Aware Human Intervention via EXECUTE-ASK-REFUSE Routingcited by 0
  27. 2026Defining AI-Native Systems: Autonomy as Revision Authoritycited by 0
  28. 2026Agentic AI Autonomy Assessment: A Decision-Support Framework Towards Governed Supply Chain Systemscited by 0
  29. 2026Regret Dominates Surprise: Design-Time Requirements Engineering for Agentic-AI Safetycited by 0
  30. 2026Anticipatory Human Oversight of Agentic AI: A Philosophical Accountcited by 0
  31. 2026Forms of LLM-Integrated Applications from LLM-Chats to Autonomous AI Agent Systemcited by 0