mlsanityData to 7 Oct 2026

CUSP: Decomposable Collective Uncertainty for Multi-Agent Multimodal Reasoning

arXiv 2609.05708 ↗First posted 4 Sep 2026cs.AI cs.LG cs.MAcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Aggregating heterogeneous vision-language models (VLMs) can improve multimodal reasoning, but neither an individual model's confidence nor that of the aggregated answer measures reliability at the system level. We present CUSP (Collective Uncertainty through Semantic Opinion Pooling), a training-free uncertainty quantification framework that maps multiple VLM responses to a shared semantic response space, pools them into a pooled semantic opinion, and reports two complementary system-level signals: collective uncertainty, the dispersion of the pooled opinion, and Jensen-Shannon divergence (JSD), the conflict among the model-level opinions. Within this pooled semantic opinion, the unnormalized collective entropy decomposes exactly into the mean of the models' individual semantic entropies and the JSD, separating total dispersion from model conflict. Requiring neither token logits nor calibration labels, CUSP applies to open-weight and commercial VLMs alike. In static multi-VLM ensembles, collective uncertainty is the strongest signal in the small-model regime (0.764 AUROC for prediction-error detection, 0.889 AUARC for abstention), outperforming uncertainty baselines majority voting and naive selection by 4.7 to 15.8 points and widening its margin as the ensemble grows; JSD is strongest in the evaluated commercial regime (0.819 AUROC, 0.910 AUARC) and ranks hard-answer model conflict with AUROC up to 0.982. The pooled prediction also improves accuracy over the average single model by 5.6 to 13.0 points. Over the full trajectory of a multi-step, multi-agent system, subagent collective uncertainty ranks system failures above chance (0.619 AUROC) and gives the best abstention ordering among the evaluated signals (0.699 AUARC).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 62 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

23 of 62 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Seeing and Reasoning with Confidence: Supercharging Multimodal LLMs with an Uncertainty-Aware Agentic Framework11 Mar
  2. 2024Large Multimodal Agents: A Survey23 Feb

Other references · 21

  1. 2025Qwen2.5-VL Technical Reportcited by 5,418
  2. 2025Qwen3-VL Technical Reportcited by 2,534
  3. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  4. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  5. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  6. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  7. 2024More Agents Is All You Needcited by 174
  8. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 159
  9. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  10. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  11. 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimationcited by 71
  12. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
9 more
  1. 2024Logits of API-Protected LLMs Leak Proprietary Informationcited by 38
  2. 2025UProp: Investigating the Uncertainty Propagation of LLMs in Multi-Step Agentic Decision-Makingcited by 36
  3. 2024Towards Uncertainty-Aware Language Agentcited by 29
  4. 2025Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Informationcited by 25
  5. 2024Reference-free Hallucination Detection for Large Vision-Language Modelscited by 23
  6. 2024Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Modelscited by 17
  7. 2026Agentic Uncertainty Quantificationcited by 16
  8. 2024Enhancing Vision-Language Model Reliability with Uncertainty-Guided Dropout Decodingcited by 14
  9. 2025Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCRcited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SCoOP: Semantic Consistent Opinion Pooling for Uncertainty Quantification in Multiple Vision-Language Model Systems
  2. 2025The Art of Saying "Maybe": A Conformal Lens for Uncertainty Benchmarking in VLMs
  3. 2026FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertainty
  4. 2026CoMet: Context and Multiplicity Decomposition for Multimodal Uncertainty Estimation
  5. 2026CoE: Collaborative Entropy for Uncertainty Quantification in Agentic Multi-LLM Systems
  6. 2025Uncertainty-o: One Model-agnostic Framework for Unveiling Uncertainty in Large Multimodal Models
  7. 2026Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volume
  8. 2026Dissecting Training-Free Uncertainty Estimation in Multimodal Large Language Models