Accountable Agents in Software Engineering: An Analysis of Terms of Service and a Research Roadmap
Not on the map. Why?
Abstract (arXiv)
AI coding assistants and autonomous agents are becoming integral to software development workflows, reshaping how code is produced, reviewed, and maintained. While recent research has focused mainly on the capabilities and impacts of productivity of these systems, much less attention has been paid to accountability: who is responsible when agents generate, modify, or recommend code? In practice, accountability is defined through the Terms of Service (ToS) and related policy documents that govern the use of AI-powered development tools. In this vision paper, we present a comparative analysis of the Terms of Service for widely used AI coding assistants and agent-enabled development tools. We examine how these documents allocate ownership, responsibility, liability, and disclosure obligations between tool providers and software developers, and we identify common patterns and divergences between providers. Our analysis reveals a consistent tendency to shift responsibility for correctness, safety, and legal compliance onto users, as well as substantial variation in how providers address issues such as indemnification, data reuse, and acceptable use. Based on these findings, we argue that existing policy frameworks are poorly aligned with increasingly agent-mediated and autonomous software development workflows. We outline a research roadmap for accountable agents in software engineering, identifying challenges and opportunities for modeling responsibility, designing governance artifacts, developing tooling that supports accountability, and conducting empirical studies of developers' perceptions and practices.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 21 of its 41 references that are in the map. First posted on arXiv 6 May 2026.
Builds on
21 of 41 references.
References · 21
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2018Datasheets for Datasetscited by 1,371
- 2018Model Cards for Model Reportingcited by 1,068
- 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
- 2024AutoCodeRover: Autonomous Program Improvementcited by 272
- 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 264
- 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 216
- 2022Do Users Write More Insecure Code with AI Assistants?cited by 186
- 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Aheadcited by 155
- 2022GitHub Copilot AI pair programmer: Asset or Liability?cited by 150
- 2022Do Language Models Plagiarize?cited by 72
- 2022Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?cited by 63
9 more
- 2024Using AI-Based Coding Assistants in Practice: State of Affairs, Perceptions, and Ways Forwardcited by 44
- 2020Principles to Practices for Responsible AI: Closing the Gapcited by 25
- 2024Transforming Software Development: Evaluating the Efficiency and Challenges of GitHub Copilot in Real-World Projectscited by 18
- 2025How Developers Interact with AI: A Taxonomy of Human-AI Collaboration in Software Engineeringcited by 14
- 2024Artificial-Intelligence Generated Code Considered Harmful: A Road Map for Secure and High-Quality Code Generationcited by 11
- 2023Exploring the Problems, their Causes and Solutions of AI Pair Programming: A Study on GitHub and Stack Overflowcited by 10
- 2025Accountability Capture: How Record-Keeping to Support AI Transparency and Accountability (Re)shapes Algorithmic Oversightcited by 5
- 2024Bot-Driven Development: From Simple Automation to Autonomous Software Development Botscited by 2
- 2025Socio-technical aspects of Agentic AIcited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Governed AI-Assisted Engineering: Graduated Human Oversight for Agentic Code Generation in Regulated Domainscited by 2
- 2026Where Accountability Lives: Mapping Human Responsibility to Workflow Artifacts in Agentic Software Developmentcited by 1
- 2026Who Finishes the Job? A Study of Follow-Up Fixes and Commit Authorship on AI Coding Agent Pull Requestscited by 0
- 2026Who Finishes the Job? A Study of Follow-Up Fixes and Commit Authorship on AI Coding Agent Pull Requests22 Sep 2026
- 2026Where Accountability Lives: Mapping Human Responsibility to Workflow Artifacts in Agentic Software Development16 Aug 2026
- 2026Governed AI-Assisted Engineering: Graduated Human Oversight for Agentic Code Generation in Regulated Domains21 Jun 2026