# Every Response Counts: Quantifying Uncertainty of LLM-based Multi-Agent Systems through Tensor Decomposition

arXiv: https://arxiv.org/abs/2604.08708 · HTML: https://mlsanity.com/paper/2604.08708/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 34 of its 49 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 34)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning](https://mlsanity.com/paper/2311.10537/) (2023)
- [Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthiness](https://mlsanity.com/paper/2308.16175/) (2023)
- [Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling](https://mlsanity.com/paper/2311.08718/) (2023)
- [A Survey of Uncertainty Estimation Methods on Large Language Models](https://mlsanity.com/paper/2503.00172/) (2025)
- [Open-TI: Open Traffic Intelligence with Augmented Language Model](https://mlsanity.com/paper/2401.00211/) (2023)
- [MoreHopQA: More Than Multi-hop Reasoning](https://mlsanity.com/paper/2406.13397/) (2024)
- [L2MAC: Large Language Model Automatic Computer for Extensive Code Generation](https://mlsanity.com/paper/2310.02003/) (2023)
- [Understanding the Uncertainty of LLM Explanations: A Perspective Based on Reasoning Topology](https://mlsanity.com/paper/2502.17026/) (2025)
- [AnyMAC: Cascading Flexible Multi-Agent Collaboration via Next-Agent Prediction](https://mlsanity.com/paper/2506.17784/) (2025)
- [LLM Uncertainty Quantification through Directional Entailment Graph and Claim Level Response Augmentation](https://mlsanity.com/paper/2407.00994/) (2024)
- [CoMAL: Collaborative Multi-Agent Large Language Models for Mixed-Autonomy Traffic](https://mlsanity.com/paper/2410.14368/) (2024)
- [Uncertainty Quantification of Large Language Models through Multi-Dimensional Responses](https://mlsanity.com/paper/2502.16820/) (2025)
- [The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Padding](https://mlsanity.com/paper/2110.10221/) (2021)
- [SAUP: Situation Awareness Uncertainty Propagation on LLM Agent](https://mlsanity.com/paper/2412.01033/) (2024)
- [SPARTan: Scalable PARAFAC2 for Large & Sparse Data](https://mlsanity.com/paper/1703.04219/) (2017)

## Built on by (most cited first, 7 of 7)

- [Counterfactual Graph for Multi-Agent LLM Calibration](https://mlsanity.com/paper/2605.30653/) (2026)
- [Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution](https://mlsanity.com/paper/2605.19228/) (2026)
- [MASkills: Continual Skills Optimization for Multi-Agent LLM Systems](https://mlsanity.com/paper/2609.02094/) (2026)
- [Position: Uncertainty Quantification in LLMs is Just Unsupervised Clustering](https://mlsanity.com/paper/2605.19220/) (2026)
- [PropUQ-MAS: Propagation-Aware Uncertainty Quantification for LLM Multi-Agent Systems](https://mlsanity.com/paper/2608.22130/) (2026)
- [Sequential Probabilistic Uncertainty Estimation for Parallel Multi-Agent Reasoning Systems](https://mlsanity.com/paper/2610.08901/) (2026)
- [MAS-Gate: Unlocking Multi-Agent System Lifecycle via Node-Centric Quantification](https://mlsanity.com/paper/2610.17613/) (2026)

## Abstract (arXiv)

While Large Language Model-based Multi-Agent Systems (MAS) consistently outperform single-agent systems on complex tasks, their intricate interactions introduce critical reliability challenges arising from communication dynamics and role dependencies. Existing Uncertainty Quantification methods, typically designed for single-turn outputs, fail to address the unique complexities of the MAS. Specifically, these methods struggle with three distinct challenges: the cascading uncertainty in multi-step reasoning, the variability of inter-agent communication paths, and the diversity of communication topologies. To bridge this gap, we introduce MATU, a novel framework that quantifies uncertainty through tensor decomposition. MATU moves beyond analyzing final text outputs by representing entire reasoning trajectories as embedding matrices and organizing multiple execution runs into a higher-order tensor. By applying tensor decomposition, we disentangle and quantify distinct sources of uncertainty, offering a comprehensive reliability measure that is generalizable across different agent structures. We provide comprehensive experiments to show that MATU effectively estimates holistic and robust uncertainty across diverse tasks and communication topologies.