mlsanityData to 14 Oct 2026

How Much Coordination Gain Is Real? A Paired Noise-Floor Protocol for Multi-Agent LLM Benchmarks

arXiv 2606.20695 ↗First posted 15 Jun 2026cs.MA cs.AIcited by 2

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Multi-agent LLM coordination papers report small benchmark deltas as evidence that one architecture beats another. A prior question: how much paired trial-0 disagreement do two protocols produce on the same model and benchmark when their API inputs are configuration-equivalent (matched by code inspection plus a SHA-256 byte audit), short of full identity-replay? On Claude Haiku 4.5 against tau^2-bench retail, the clean configuration-equivalent contrast (no_coord vs. intercept, both inert at trial 0) gives signed paired gaps of +10pp and 0pp across two n=100 seeds; pooled across both, +5pp with Wilson CI [-2,+12], not significant. The largest single-seed contrast (+18pp pull-vs-intercept, p_corr=0.012) did not reproduce at the second seed (-3pp, p_corr=1.0); no trial-0 contrast is significant after Bonferroni at either seed or pooled. The envelope of observed paired gaps spans [-3,+18]pp across two seeds, with pooled upper Wilson CI ~15pp. Seven of ten recent multi-agent coordination architectures report headline effects below this local floor, and one more sits inside the envelope; whether they survive a same-model paired replication is, by construction, untested in their original settings. We define coordination-active pass^k, pass^k restricted to trials where the coordination mechanism is logically active, as the minimum reporting protocol, with sample-size targets and runtime hooks in the body. Measurements run on ET-MCP, a task-scoped negative-knowledge store conformant with MCP 2026-07-28, used as a substrate to isolate reader-side choices, not as a contribution. On Haiku 4.5 the candidate readers (pull, intercept) do not improve trial-1 recovery; we give a preliminary diagnosis of failure modes with refinements on existing production hook surfaces.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 14 of its 36 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

14 of 36 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Condition11 Feb
  2. 2026Agent Drift: Quantifying Behavioral Degradation in Multi-Agent LLM Systems Over Extended Interactions7 Jan
  3. 2025Analyzing Information Sharing and Coordination in Multi-Agent Planning18 Aug
  4. 2023ExpeL: LLM Agents Are Experiential Learners20 Aug

Other references · 10

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  2. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  3. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  4. 2026Towards a Science of AI Agent Reliabilitycited by 65
  5. 2026ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditionscited by 36
  6. 2025Terrarium: Revisiting the Blackboard for Multi-Agent Safety, Privacy, and Security Studiescited by 10
  7. 2026AgentTrace: Causal Graph Tracing for Root Cause Analysis in Deployed Multi-Agent Systemscited by 7
  8. 2026CodeDelegator: Mitigating Context Pollution via Role Separation in Code-as-Action Agentscited by 4
  9. 2026Automated Self-Testing as a Quality Gate: Evidence-Driven Release Management for LLM Applicationscited by 3
  10. 2026Enhancing Model Context Protocol (MCP) with Context-Aware Server Collaborationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026DPBench: Structural Determinants of Multi-Agent LLM Coordination Under Simultaneous Resource Contention
  2. 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents
  3. 2026When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines
  4. 2026Benchmarking Emergent Coordination in Large-Scale LLM Populations: An Evaluation Framework on the MoltBook Archive
  5. 2026"LLM Agent Performance" Is Not a Single Evaluation Target
  6. 2026Counterfactual Graph for Multi-Agent LLM Calibration
  7. 2026MASBench: Benchmarking LLM-based Multi-Agent Collaboration under Partial Observability
  8. 2026Oracle Gap and Signal Fidelity: A Fixed-Pool Diagnostic for Test-Time Collaboration