mlsanityData to 7 Oct 2026

SCoOP: Semantic Consistent Opinion Pooling for Uncertainty Quantification in Multiple Vision-Language Model Systems

arXiv 2603.23853 ↗First posted 25 Mar 2026cs.AI cs.MAcited by 0

On the map: LLM reasoning → Hallucination detection and mitigation

also in 1 more
Abstract (arXiv)

Combining multiple Vision-Language Models (VLMs) can enhance multimodal reasoning and robustness, but aggregating heterogeneous models' outputs amplifies uncertainty and increases the risk of hallucinations. We propose SCoOP (Semantic-Consistent Opinion Pooling), a training-free uncertainty quantification (UQ) framework for multi-VLM systems through uncertainty-weighted linear opinion pooling. The core idea is to treat each VLM as a probabilistic "expert," sample multiple outputs, map them to a unified space, aggregate their opinions, and produce a system-level uncertainty score. Unlike prior UQ methods designed for single models, SCoOP explicitly measures collective, system-level uncertainty across multiple VLMs, enabling effective hallucination detection and abstention for highly uncertain samples. On ScienceQA, SCoOP achieves an AUROC of 0.866 for hallucination detection, outperforming baselines (0.732-0.757) by approximately 10-13%. For abstention, it attains an AURAC of 0.907, exceeding baselines (0.818-0.840) by 7-9%. Despite these gains, SCoOP introduces only microsecond-level aggregation overhead relative to the baselines, which is trivial compared to typical VLM inference time (on the order of seconds). These results demonstrate that SCoOP provides an efficient and principled mechanism for uncertainty-aware aggregation, advancing the reliability of multimodal AI systems. Our code is publicly available at https://github.com/chungenyu6/SCoOP.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 30 of its 40 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

30 of 40 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Verify when Uncertain: Beyond Self-Consistency in Black Box Hallucination Detection20 Feb

Other references · 29

  1. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  3. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  4. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  5. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  6. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 352
  7. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  8. 2024More Agents Is All You Needcited by 174
  9. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 159
  10. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  11. 2024A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directionscited by 115
  12. 2024Large Multimodal Agents: A Surveycited by 99
17 more
  1. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
  2. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  3. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedbackcited by 80
  4. 2024VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimationcited by 71
  5. 2024API Is Enough: Conformal Prediction for Large Language Models Without Logit-Accesscited by 68
  6. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
  7. 2022MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Modelcited by 41
  8. 2024Logits of API-Protected LLMs Leak Proprietary Informationcited by 38
  9. 2024Towards Uncertainty-Aware Language Agentcited by 29
  10. 2025Beyond Majority Voting: LLM Aggregation by Leveraging Higher-Order Informationcited by 25
  11. 2024Reference-free Hallucination Detection for Large Vision-Language Modelscited by 23
  12. 2024Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Modelscited by 17
  13. 2025Seeing and Reasoning with Confidence: Supercharging Multimodal LLMs with an Uncertainty-Aware Agentic Frameworkcited by 12
  14. 2025Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCRcited by 12
  15. 2025Calibrating Uncertainty Quantification of Multi-Modal LLMs using Groundingcited by 8
  16. 2025Predictive Auditing of Hidden Tokens in LLM APIs via Reasoning Length Estimationcited by 7
  17. 2024Enhancing Multi-Agent Consensus through Third-Party LLM Integration: Analyzing Uncertainty and Mitigating Hallucinations in Large Language Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Toward More Reliable Artificial Intelligence: Reducing Hallucinations in Vision-Language Models
  2. 2026UHP Detection: LVLMs have their Unique Hallucination Pattern in the Consistency Space
  3. 2025Revisiting Hallucination Detection with Effective Rank-based Uncertainty
  4. 2025Visual hallucination detection in large vision-language models via evidential conflict
  5. 2025SegSub: Evaluating Robustness to Knowledge Conflicts and Hallucinations in Vision-Language Models
  6. 2026Segmentation-Based Attention Entropy: Detecting and Mitigating Object Hallucinations in Large Vision-Language Models
  7. 2025Leveraging NTPs for Efficient Hallucination Detection in VLMs
  8. 2025Beyond Single Models: Mitigating Multimodal Hallucinations via Adaptive Token Ensemble Decoding