mlsanityData to 14 Oct 2026

The Perils of Agency: How Developers Perceive, Prioritize, and Address Risks in Agentic AI Products

arXiv 2606.15485 ↗First posted 13 Jun 2026cs.CY cs.AI cs.HCcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Agentic AI systems act autonomously, use tools, adapt to context, and operate in complex real-world environments. However, these same characteristics can create or exacerbate product risks. We studied how industry developers (n=35) perceive, prioritize, and address the risks in their agentic AI products. We found that developers' perceptions of risk were closely tied to the qualities that made the product agentic, such as autonomy, tool use, and usage in a real-world context. Developers prioritized product and business risks before considering downstream societal risks like job displacement and end-user privacy. This prioritization also impacted developers' ability and motivation to mitigate agentic risks. Finally, developers lacked mature controls for containing agentic risks, often relying on constraining the same characteristics that make agents useful: e.g., autonomy and goal complexity. These findings reveal a capability vs. risk control tension in agentic AI development: developers need to address risks that emerge from agentic capabilities, yet they currently have limited support for doing so without constraining agentic functionality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 13 of its 47 references that are in the map. First posted on arXiv 13 Jun 2026.

Builds on

13 of 47 references.

References · 13

  1. 2018Datasheets for Datasetscited by 1,386
  2. 2018Model Cards for Model Reportingcited by 1,080
  3. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 548
  4. 2021Trustworthy AI: From Principles to Practicescited by 207
  5. 2023Agent Instructs Large Language Models to be General Zero-Shot Reasonerscited by 32
  6. 2023Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Riskscited by 31
  7. 2025Characterizing AI Agents for Alignment and Governancecited by 30
  8. 2023AHA!: Facilitating AI Impact Assessment by Generating Examples of Harmscited by 26
  9. 2021AI Ethics Principles in Practice: Perspectives of Designers and Developerscited by 12
  10. 2023Ethics in the Age of AI: An Analysis of AI Practitioners' Awareness and Challengescited by 6
  11. 2025A Risk Ontology for Evaluating AI-Powered Psychotherapy Virtual Agentscited by 5
  12. 2025Why Johnny Can't Use Agents: Industry Aspirations vs. User Realities with AI Agentscited by 5
1 more
  1. 2025Privy: Envisioning and Mitigating Privacy Risks for Consumer-facing AI Product Conceptscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025With Great Capabilities Come Great Responsibilities: Introducing the Agentic Risk & Capability Framework for Governing Agentic AI Systems
  2. 2025Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025
  3. 2026ParanoiaEval: Benchmarking Unnecessary Defensive Work in Agentic Coding
  4. 2026Agentic Explainability at Scale: Between Corporate Fears and XAI Needs
  5. 2026Agentic AI in Engineering and Manufacturing: Industry Perspectives on Utility, Adoption, Challenges, and Opportunities
  6. 2026Risk Architecture for AI-Native Engineering Teams: An Organizational Framework for Agentic System Governance
  7. 2026Are We All Using Agents the Same Way? An Empirical Study of Core and Peripheral Developers Use of Coding Agents
  8. 2026Assistance to Autonomy: A Systematic Literature Review of Agentic AI across the Software Development Life Cycle