# Beyond Perfect Teammates: Multi-Agent Collaboration with Unreliable Colleagues arXiv: https://arxiv.org/abs/2610.16117 ยท HTML: https://mlsanity.com/paper/2610.16117/ ## Key facts - No citing papers in the map yet (data through 14 Oct 2026). - Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. - Builds on 27 of its 35 references that are in the map. - First posted on arXiv 12 Oct 2026. ## Placed in - LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/) ## Builds on (most cited first, 27 of 27) - [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023) - [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023) - [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023) - [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023) - [ChatDev: Communicative Agents for Software Development](https://mlsanity.com/paper/2307.07924/) (2023) - [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025) - [AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors](https://mlsanity.com/paper/2308.10848/) (2023) - [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026) - [Benchmarking Large Language Models in Retrieval-Augmented Generation](https://mlsanity.com/paper/2309.01431/) (2023) - [Gemma 4 Technical Report](https://mlsanity.com/paper/2607.02770/) (2026) - [MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents](https://mlsanity.com/paper/2503.01935/) (2025) - [FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions](https://mlsanity.com/paper/2310.15421/) (2023) - [AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems?](https://mlsanity.com/paper/2509.03312/) (2025) - [On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents](https://mlsanity.com/paper/2408.00989/) (2024) - [Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence](https://mlsanity.com/paper/2407.07061/) (2024) - [PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks](https://mlsanity.com/paper/2411.00081/) (2024) - [OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflows](https://mlsanity.com/paper/2508.09124/) (2025) - [Agent-Oriented Planning in Multi-Agent Systems](https://mlsanity.com/paper/2410.02189/) (2024) - [MCPEval: Automatic MCP-based Deep Evaluation for AI Agent Models](https://mlsanity.com/paper/2507.12806/) (2025) - [Tools Fail: Detecting Silent Errors in Faulty Tools](https://mlsanity.com/paper/2406.19228/) (2024) - [AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Condition](https://mlsanity.com/paper/2602.11348/) (2026) - [Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs](https://mlsanity.com/paper/2505.11556/) (2025) - [Autonomous Agents for Collaborative Task under Information Asymmetry](https://mlsanity.com/paper/2406.14928/) (2024) - [When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification](https://mlsanity.com/paper/2602.11199/) (2026) - [MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2602.19843/) (2026) - [Graph-of-Agents: A Graph-based Framework for Multi-Agent LLM Collaboration](https://mlsanity.com/paper/2604.17148/) (2026) - [Hell or High Water: Evaluating Agentic Recovery from External Failures](https://mlsanity.com/paper/2508.11027/) (2025) ## Abstract (arXiv) LLM-based multi-agent collaboration is increasingly used to solve complex tasks by dividing work among specialized agents and integrating complementary expertise. Yet existing systems and evaluations largely rest on an implicit perfect-colleague assumption: collaborators provide correct and complete information and respond promptly to requests. In realistic teamwork, however, collaborators may return information that does not answer the intended question, provide insufficient detail to support a decision, or fail to respond to a request. To separate task-solving competence from the ability to recover an unreliable evidence-gathering process, we introduce the Unreliable Colleagues Benchmark (UnCoBench), which organizes behavioral failures through Misaligned, Incomplete, and Missed settings. The main agent need to verify contextual correctness, diagnose information gap and track unresolved dependencies in order to obtain the correct evidence for solving the task. We also introduce a verifiable and scalable automated generation pipeline that pairs LLM-based scenario generation with executable checks of answer uniqueness and role necessity. Results across ten models show that similar reliable-setting performance can result in different robustness to unreliable evidence. Trace analysis reveals frequent omissions of corrective follow-up, and the recovery behavior involves inefficient waiting and redundant queries. These findings show that robust collaboration requires not only sound reasoning, but also targeted action to diagnose evidence failures, recover relevant information, and integrate it into decisions. Our data and source code are available at https://github.com/iNLP-Lab/UnCoBench.