mlsanityData to 7 Oct 2026

DiLLS: Interactive Diagnosis of LLM-based Multi-agent Systems via Layered Summary of Agent Behaviors

arXiv 2602.05446 ↗First posted 5 Feb 2026cs.HCcited by 7

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Large language model (LLM)-based multi-agent systems have demonstrated impressive capabilities in handling complex tasks. However, the complexity of agentic behaviors makes these systems difficult to understand. When failures occur, developers often struggle to identify root causes and to determine actionable paths for improvement. Traditional methods that rely on inspecting raw log records are inefficient, given both the large volume and complexity of data. To address this challenge, we propose a framework and an interactive system, DiLLS, designed to reveal and structure the behaviors of multi-agent systems. The key idea is to organize information across three levels of query completion: activities, actions, and operations. By probing the multi-agent system through natural language, DiLLS derives and organizes information about planning and execution into a structured, multi-layered summary. Through a user study, we show that DiLLS significantly improves developers' effectiveness and efficiency in identifying, diagnosing, and understanding failures in LLM-based multi-agent systems.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 34 of its 63 references that are in the map. First posted on arXiv 5 Feb 2026.

Builds on

34 of 63 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Interactive Debugging and Steering of Multi-Agent AI Systems3 Mar
  2. 2025Autonomous Legacy Web Application Upgrades Using a Multi-Agent System31 Jan
  3. 2024AutoGen Studio: A No-Code Developer Tool for Building and Debugging Multi-Agent Systems9 Aug
  4. 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead7 Apr
  5. 2024CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology2 Feb
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug

Other references · 28

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  3. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  4. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  5. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  6. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  7. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  8. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
  9. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
  10. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 203
  11. 2024Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferencescited by 142
  12. 2024Empowering Biomedical Discovery with AI Agentscited by 117
16 more
  1. 2024Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffoldingcited by 115
  2. 2023Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?cited by 104
  3. 2023Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generationcited by 103
  4. 2025AgentNet: Decentralized Evolutionary Coordination for LLM-based Multi-Agent Systemscited by 86
  5. 2024AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planningcited by 81
  6. 2023EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteriacited by 73
  7. 2024ProtAgents: Protein discovery via large language model multi-agent collaborations combining physics and machine learningcited by 63
  8. 2023Rehearsal: Simulating Conflict to Teach Conflict Resolutioncited by 54
  9. 2025Multi-Agent Systems Execute Arbitrary Malicious Codecited by 52
  10. 2024MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPscited by 45
  11. 2023Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Educationcited by 38
  12. 2023Memory Sandbox: Transparent and Interactive Memory Management for Conversational Agentscited by 36
  13. 2024Interactive Prompt Debugging with Sequence Saliencecited by 6
  14. 2024A New Era in Human Factors Engineering: A Survey of the Applications and Prospects of Large Multimodal Modelscited by 5
  15. 2025Disaster Management in the Era of Agentic AI Systems: A Vision for Collective Human-Machine Intelligence for Augmented Resiliencecited by 4
  16. 2025Closed-Loop Vision-Language Planning for Multi-Agent Coordinationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 5
  2. 2026Towards Security-Auditable LLM Agents: A Unified Graph Representationcited by 4
  3. 2026"What Did It Actually Do?": Understanding Risk Awareness and Traceability for Computer-Use Agentscited by 2
  4. 2026InconLens: Interactive Visual Diagnosis of Behavioral Inconsistencies in LLM-based Agentic Systemscited by 1
  5. 2026Within the MDT Room: Situated in Multidisciplinary Team-Grounded Agent Debate for Clinical Diagnosiscited by 0
  6. 2026VeriLLMed: Interactive Visual Debugging of Medical Large Language Models with Knowledge Graphscited by 0
  7. 2026MUSE: An Interactive Meta-Agent for Understanding and Steering LLM-powered Data Science Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LLM-Based Agentic Systems for Software Engineering: Challenges and Opportunities
  2. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  3. 2026Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems
  4. 2026Tangent: An Empirical Study of Testing Practices for LLM-Based Agent Applications
  5. 2025DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems
  6. 2026Agentic CLEAR: Automating Multi-Level Evaluation of LLM Agents
  7. 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems
  8. 2025Where LLM Agents Fail and How They can Learn From Failures