# Automated structural testing of LLM-based agents: methods, framework, and case studies arXiv: https://arxiv.org/abs/2601.18827 ยท HTML: https://mlsanity.com/paper/2601.18827/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. - Builds on 33 of its 80 references that are in the map. - First posted on arXiv 25 Jan 2026. ## Placed in - LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/) ## Builds on (most cited first, 30 of 33) - [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020) - [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022) - [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023) - [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023) - [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023) - [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022) - [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024) - [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023) - [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024) - [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023) - [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025) - [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025) - [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024) - [A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models](https://mlsanity.com/paper/2401.01313/) (2024) - [Survey on Evaluation of LLM-based Agents](https://mlsanity.com/paper/2503.16416/) (2025) - [Evil Geniuses: Delving into the Safety of LLM-based Agents](https://mlsanity.com/paper/2311.11855/) (2023) - [A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models](https://mlsanity.com/paper/2402.13457/) (2024) - [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024) - [Cognitive Mirage: A Review of Hallucinations in Large Language Models](https://mlsanity.com/paper/2309.06794/) (2023) - [LLMs Will Always Hallucinate, and We Need to Live With This](https://mlsanity.com/paper/2409.05746/) (2024) - [TrustAgent: Towards Safe and Trustworthy LLM-based Agents](https://mlsanity.com/paper/2402.01586/) (2024) - [One Token to Fool LLM-as-a-Judge](https://mlsanity.com/paper/2507.08794/) (2025) - [Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts](https://mlsanity.com/paper/2209.12711/) (2022) - [Test-Driven Development for Code Generation](https://mlsanity.com/paper/2402.13521/) (2024) - [Defining and Detecting the Defects of the Large Language Model-based Autonomous Agents](https://mlsanity.com/paper/2412.18371/) (2024) - [Software Testing of Generative AI Systems: Challenges and Opportunities](https://mlsanity.com/paper/2309.03554/) (2023) - [Automatic Creation of Acceptance Tests by Extracting Conditionals from Requirements: NLP Approach and Case Study](https://mlsanity.com/paper/2202.00932/) (2022) - [Open Tracing Tools: Overview and Critical Comparison](https://mlsanity.com/paper/2207.06875/) (2022) - [Generative AI for Test Driven Development: Preliminary Results](https://mlsanity.com/paper/2405.10849/) (2024) - [Software Test Automation Maturity -- A Survey of the State of the Practice](https://mlsanity.com/paper/2004.09210/) (2020) ## Built on by (most cited first, 3 of 3) - [Understanding Agent-Reactive Bugs at the Model-Harness Boundary: An Empirical Study of LLM Agent Issue Reports](https://mlsanity.com/paper/2607.15684/) (2026) - [Assistance to Autonomy: A Systematic Literature Review of Agentic AI across the Software Development Life Cycle](https://mlsanity.com/paper/2605.15245/) (2026) - [Chronicle: Cut-Point Replay for Regression Testing of LLM Agents](https://mlsanity.com/paper/2609.20625/) (2026) ## Abstract (arXiv) LLM-based agents are rapidly being adopted across diverse domains. Since they interact with users without supervision, they must be tested extensively. Current testing approaches focus on acceptance-level evaluation from the user's perspective. While intuitive, these tests require manual evaluation, are difficult to automate, do not facilitate root cause analysis, and incur expensive test environments. In this paper, we present methods to enable structural testing of LLM-based agents. Our approach utilizes traces (based on OpenTelemetry) to capture agent trajectories, employs mocking to enforce reproducible LLM behavior, and adds assertions to automate test verification. This enables testing agent components and interactions at a deeper technical level within automated workflows. We demonstrate how structural testing enables the adaptation of software engineering best practices to agents, including the test automation pyramid, regression testing, test-driven development, and multi-language testing. In representative case studies, we demonstrate automated execution and faster root-cause analysis. Collectively, these methods reduce testing costs and improve agent quality through higher coverage, reusability, and earlier defect detection. We provide an open source reference implementation on GitHub.