# Counterfactual Graph for Multi-Agent LLM Calibration

arXiv: https://arxiv.org/abs/2605.30653 · HTML: https://mlsanity.com/paper/2605.30653/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 28 May 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 28 of 28)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Verified Uncertainty Calibration](https://mlsanity.com/paper/1909.10155/) (2019)
- [Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models](https://mlsanity.com/paper/2310.06117/) (2023)
- [Large Language Models as Analogical Reasoners](https://mlsanity.com/paper/2310.01714/) (2023)
- [Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models](https://mlsanity.com/paper/2503.02623/) (2025)
- [AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answering](https://mlsanity.com/paper/2510.05445/) (2025)
- [Confidence Calibration and Rationalization for LLMs via Multi-Agent Deliberation](https://mlsanity.com/paper/2404.09127/) (2024)
- [SteerConf: Steering LLMs for Confidence Elicitation](https://mlsanity.com/paper/2503.02863/) (2025)
- [EvolveRouter: Co-Evolving Routing and Prompt for Multi-Agent Question Answering](https://mlsanity.com/paper/2604.05149/) (2026)
- [Graph is a Substrate Across Data Modalities](https://mlsanity.com/paper/2601.22384/) (2026)
- [Rethinking LLM Uncertainty: A Multi-Agent Approach to Estimating Black-Box Model Uncertainty](https://mlsanity.com/paper/2412.09572/) (2024)
- [NG-Router: Graph-Supervised Multi-Agent Collaboration for Nutrition Question Answering](https://mlsanity.com/paper/2510.09854/) (2025)
- [Same Signal, Opposite Meaning: Direction-Informed Adaptive Learning for LLM Agents](https://mlsanity.com/paper/2605.06908/) (2026)
- [Every Response Counts: Quantifying Uncertainty of LLM-based Multi-Agent Systems through Tensor Decomposition](https://mlsanity.com/paper/2604.08708/) (2026)
- [Epistemic Gain, Aleatoric Cost: Uncertainty Decomposition in Multi-Agent Debate for Math Reasoning](https://mlsanity.com/paper/2603.01221/) (2026)
- [DiscoUQ: Structured Disagreement Analysis for Uncertainty Quantification in LLM Agent Ensembles](https://mlsanity.com/paper/2603.20975/) (2026)

## Built on by (most cited first, 7 of 7)

- [Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLM](https://mlsanity.com/paper/2607.26773/) (2026)
- [Agent Behavioral Contracts II: Certifying Compositional Reliability Without Assuming Independence](https://mlsanity.com/paper/2608.12895/) (2026)
- [Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Study](https://mlsanity.com/paper/2609.07395/) (2026)
- [Target-Checked Reliability Score Refinement for Video Question Answering](https://mlsanity.com/paper/2609.13288/) (2026)
- [Social Circuits behind Multi-agent Echo Chambers](https://mlsanity.com/paper/2609.34444/) (2026)
- [CLEAT: Learning Value-Guided Clarification for Interactive Agents](https://mlsanity.com/paper/2610.12655/) (2026)
- [RIVO: Lightweight Adaptive Rerouting for Deep Research Agents](https://mlsanity.com/paper/2610.15910/) (2026)

## Abstract (arXiv)

Multi-agent LLM systems often treat agreement as evidence: when many agents in a panel give the same answer, that answer is assumed to be more reliable. We show that this assumption can fail after agents communicate. Communication can induce correlated failures and false consensus, so the same vote share may reflect reliable agreement in one topology but over-confidence in another. We propose CAGE-CAL, a counterfactual agent-graph calibration framework for multi-agent LLMs. For each query, CAGE-CAL compares an observed post-communication agent graph with a matched counterfactual no-communication graph, capturing both pairwise failure correlations and group-level dependencies. Rather than simply counting how many agents agree, CAGE-CAL estimates the counterfactual shift between observed and no-communication dependence, and calibrates confidence accordingly. Across five benchmarks, CAGE-CAL improves reliability discrimination with competitive ECE, and its calibrated confidence further improves topology selection over the best fixed-topology strategy.