Towards Risk-free AI Agent Deployment
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
LLM-based agents are rapidly moving from research prototypes into the core business processes of organizations, but these agents pose deployment risks to security, compliance, and functionality. In this article, we argue that risk-free deployment must be grounded in the agent's trajectory: the recorded sequence of reasoning steps, tool invocations, and environmental observations. Trajectories are available for any agent, and many failures are visible only in the trajectory. To make agents deployable and sustainable, we advocate agent testing and debugging as a systematic research direction for detecting and mitigating these risks. This article begins with the challenges of testing agents, including the oracle problem, non-determinism, trajectory validation, and the absence of adequacy metrics. We then turn to debugging agents, from automated failure attribution to repair and self-evolution. We distill these directions into a practical deployment-readiness checklist covering the full deployment lifecycle. Finally, we identify open problems, i.e., formal adequacy metrics, root-cause attribution over long-horizon trajectories, and the reliability of self-evolving agents, that the community must address to enable trustworthy agent deployment.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 22 of its 38 references that are in the map. First posted on arXiv 17 Aug 2026.
Builds on
22 of 38 references.
Closest ancestors · 11
Same direction, newest first
- 2026Tangent: An Empirical Study of Testing Practices for LLM-Based Agent Applications9 Aug
- 2026TrajAudit: Automated Failure Diagnosis for Agentic Coding Systems26 May
- 2026Wink: Recovering from Misbehaviors in Coding Agents19 Feb
- 2026AgentRx: Diagnosing AI Agent Failures from Execution Trajectories2 Feb
- 2026ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditions3 Jan
- 2025Process-Centric Analysis of Agentic Software Systems2 Dec
- 2025Agentic Property-Based Testing: Finding Bugs Across the Python Ecosystem10 Oct
- 2025Where LLM Agents Fail and How They can Learn From Failures29 Sep
- 2025Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories23 Jun
- 2025Why Do Multi-Agent LLM Systems Fail?17 Mar
- 2025Agentic AI Software Engineers: Programming with Trust19 Feb
Other references · 11
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
- 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 284
- 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 278
- 2026Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skillscited by 187
- 2020Metamorphic Testing: A New Approach for Generating Next Test Casescited by 75
- 2025SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agentscited by 69
- 2025Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agentscited by 24
- 2017Chaos Engineeringcited by 19
- 2026Model Context Protocol (MCP) Tool Descriptions Are Smelly! Towards Improving AI Agent Efficiency with Augmented MCP Tool Descriptionscited by 17
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems
- 2026AgentAssay: Token-Efficient Regression Testing for Non-Deterministic AI Agent Workflows
- 2026Automated structural testing of LLM-based agents: methods, framework, and case studies
- 2026Beyond Component Testing: Validating Agentic AI Systems
- 2025Evaluation and Benchmarking of LLM Agents: A Survey
- 2026ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failures
- 2026Agent Lifecycle Toolkit (ALTK): Reusable Middleware Components for Robust AI Agents
- 2026A Large-Scale Empirical Study of Quality Assurance Practices and Gaps in AI Agents