mlsanityData to 14 Oct 2026

BiG-SURE - Bipartite Graph for Semantic Uncertainty and Reliability Estimation of LLMs

arXiv 2608.30646 ↗First posted 31 Aug 2026cs.CL cs.AI cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Reliable uncertainty estimation is a crucial requirement for deploying large language models (LLMs) and vision-language models (VLMs) in safety-critical settings, especially when the model parameters are not accessible (black-box). We propose BiG-SURE, an uncertainty estimator based on cross-temperature semantic agreement. The method samples low-temperature responses as stable semantic anchors and high-temperature responses as probes under meaning-preserving input transformations. It then constructs an anchor-probe Bipartite Graph (BiG) using NLI-based entailment scores and defines confidence through the normalized squared spectral energy of this matrix, with uncertainty given by its complement. This bipartite graph-based Semantic Uncertainty and Reliability Estimation (SURE) score measures whether high-temperature probes remain semantically aligned with the model's stable low-temperature belief or not. We evaluate BiG-SURE on text QA, multilingual QA, and multimodal QA tasks across multiple model families. In these experiments, BiG-SURE improves average abstention AUROC over prior black-box uncertainty estimators, while remaining simple, unsupervised, and applicable to black-box model settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 25 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

14 of 25 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025FESTA: Functionally Equivalent Sampling for Trust Assessment of Multimodal LLMs20 Sep
  2. 2023Improved Baselines with Visual Instruction Tuning5 Oct

Other references · 12

  1. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  2. 2015VQA: Visual Question Answeringcited by 2,506
  3. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  4. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  5. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  6. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  7. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 211
  8. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 159
  9. 2025Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Modelscited by 44
  10. 2025Beyond Semantic Entropy: Boosting LLM Uncertainty Quantification with Pairwise Semantic Similaritycited by 28
  11. 2025Benchmarking and Confidence Evaluation of LALMs For Temporal Reasoningcited by 10
  12. 2026Semantic Self-Distillation for Language Model Uncertaintycited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantification
  2. 2025Revisiting Uncertainty Estimation and Calibration of Large Language Models
  3. 2026Semantic Calibration Prevails Where Token Confidence Fails: Benchmarking Long-Form Scientific QA
  4. 2026Dissecting Training-Free Uncertainty Estimation in Multimodal Large Language Models
  5. 2026ChainUQ: Reasoning Consistency-Aware Uncertainty Quantification for Large Language Models
  6. 2024Towards Trustworthy Knowledge Graph Reasoning: An Uncertainty Aware Perspective
  7. 2026Semantic Uncertainty Quantification Needs Factual Equivalence
  8. 2026Argument Graph Uncertainty: Quantifying Uncertainty from the Logical Structure of Reasoning Chains