mlsanityData to 14 Oct 2026

Multi-Agent LLM Orchestration Achieves Deterministic, High-Quality Decision Support for Incident Response

arXiv 2511.15755 ↗First posted 19 Nov 2025cs.AI cs.SEcited by 6

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large language models (LLMs) promise to accelerate incident response in production systems, yet single-agent approaches generate vague, unusable recommendations. We present MyAntFarm.ai, a reproducible containerized framework demonstrating that multi-agent orchestration fundamentally transforms LLM-based incident response quality. Through 348 controlled trials comparing single-agent copilot versus multi-agent systems on identical incident scenarios, we find that multi-agent orchestration achieves 100% actionable recommendation rate versus 1.7% for single-agent approaches, an 80 times improvement in action specificity and 140 times improvement in solution correctness. Critically, multi-agent systems exhibit zero quality variance across all trials, enabling production SLA commitments impossible with inconsistent single-agent outputs. Both architectures achieve similar comprehension latency (approx.40s), establishing that the architectural value lies in deterministic quality, not speed. We introduce Decision Quality (DQ), a novel metric capturing validity, specificity, and correctness properties essential for operational deployment that existing LLM metrics do not address. These findings reframe multi-agent orchestration from a performance optimization to a production-readiness requirement for LLM-based incident response. All code, Docker configurations, and trial data are publicly available for reproduction.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 6 of its 19 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

6 of 19 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Scaling Large Language Model-based Multi-Agent Collaboration11 Jun

Other references · 5

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  3. 2023ChatDev: Communicative Agents for Software Developmentcited by 919
  4. 2022Deep Learning for Time Series Anomaly Detection: A Surveycited by 131
  5. 2025A Survey of AIOps in the Era of Large Language Modelscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Utility-Guided Agent Orchestration for Efficient LLM Tool Use
  2. 2026PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting
  3. 2026ClawArena-Team: Benchmarking Subagent Orchestration and Dynamic Workflows in Language-Model Agents
  4. 2026Agent Gym: A Framework for Continuous Evaluation and Evolution of LLM Agents Through Human-in-the-Loop Feedback
  5. 2026AIR: Improving Agent Safety through Incident Response
  6. 2025Multi-Mission Tool Bench: Assessing the Robustness of LLM based Agents through Related and Dynamic Missions
  7. 2024Evaluation-Driven Development and Operations of LLM Agents: A Process Model and Reference Architecture
  8. 2025Modeling Response Consistency in Multi-Agent LLM Systems: A Comparative Analysis of Shared and Separate Context Approaches