# Consistency and Uncertainty: Identifying Unreliable Responses From Black-Box Vision-Language Models for Selective Visual Question Answering

arXiv: https://arxiv.org/abs/2404.10193 · HTML: https://mlsanity.com/paper/2404.10193/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 16 Apr 2024.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019)
- [SelectiveNet: A Deep Neural Network with an Integrated Reject Option](https://mlsanity.com/paper/1901.09192/) (2019)
- [Consistent Estimators for Learning to Defer to an Expert](https://mlsanity.com/paper/2006.01862/) (2020)
- [Black-Box Tuning for Language-Model-as-a-Service](https://mlsanity.com/paper/2201.03514/) (2022)
- [Selective Question Answering under Domain Shift](https://mlsanity.com/paper/2006.09462/) (2020)
- [Cycle Self-Training for Domain Adaptation](https://mlsanity.com/paper/2103.03571/) (2021)
- [Implicit Semantic Data Augmentation for Deep Networks](https://mlsanity.com/paper/1909.12220/) (2019)
- [Cycle-Consistency for Robust Visual Question Answering](https://mlsanity.com/paper/1902.05660/) (2019)
- [Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectly](https://mlsanity.com/paper/2204.13631/) (2022)
- [Deep Gamblers: Learning to Abstain with Portfolio Theory](https://mlsanity.com/paper/1907.00208/) (2019)
- [Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering](https://mlsanity.com/paper/2104.03149/) (2021)
- [Improving Selective Visual Question Answering by Learning from Your Peers](https://mlsanity.com/paper/2306.08751/) (2023)

## Built on by (most cited first, 30 of 32)

- [Seeing and Reasoning with Confidence: Supercharging Multimodal LLMs with an Uncertainty-Aware Agentic Framework](https://mlsanity.com/paper/2503.08308/) (2025)
- [Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volume](https://mlsanity.com/paper/2602.24195/) (2026)
- [ViQAgent: Zero-Shot Video Question Answering via Agent with Open-Vocabulary Grounding Validation](https://mlsanity.com/paper/2505.15928/) (2025)
- [The Cost of Reasoning: Chain-of-Thought Induces Overconfidence in Vision-Language Models](https://mlsanity.com/paper/2603.16728/) (2026)
- [Improving Medical Diagnostics with Vision-Language Models: Convex Hull-Based Uncertainty Analysis](https://mlsanity.com/paper/2412.00056/) (2024)
- [From Objects to Events: Unlocking Complex Visual Understanding in Object Detectors via LLM-guided Symbolic Reasoning](https://mlsanity.com/paper/2502.05843/) (2025)
- [PARC: A Quantitative Framework Uncovering the Symmetries within Vision Language Models](https://mlsanity.com/paper/2506.14808/) (2025)
- [HARMONY: Hidden Activation Representations and Model Output-Aware Uncertainty Estimation for Vision-Language Models](https://mlsanity.com/paper/2510.22171/) (2025)
- [Remember with Confidence: Uncertainty Quantification for Spatio-temporal Memory with Probabilistic Guarantees](https://mlsanity.com/paper/2606.08277/) (2026)
- [FastRM: An efficient and automatic explainability framework for multimodal generative models](https://mlsanity.com/paper/2412.01487/) (2024)
- [Entropy Heat-Mapping: Localizing GPT-Based OCR Errors with Sliding-Window Shannon Analysis](https://mlsanity.com/paper/2505.00746/) (2025)
- [Hallucination Filtering in Radiology Vision-Language Models Using Discrete Semantic Entropy](https://mlsanity.com/paper/2510.09256/) (2025)
- [Mapping User Trust in Vision Language Models: Research Landscape, Challenges, and Prospects](https://mlsanity.com/paper/2505.05318/) (2025)
- [BLaVe-CoT: Consistency-Aware Visual Question Answering for Blind and Low Vision Users](https://mlsanity.com/paper/2509.06010/) (2025)
- [CORA: Consistency-Guided Semi-Supervised Framework for Reasoning Segmentation](https://mlsanity.com/paper/2511.17755/) (2025)
- [WildIng: A Wildlife Image Invariant Representation Model for Geographical Domain Shift](https://mlsanity.com/paper/2601.00993/) (2026)
- [SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring](https://mlsanity.com/paper/2604.25855/) (2026)
- [FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertainty](https://mlsanity.com/paper/2606.14728/) (2026)
- [Bridging the Semantic Gaps: Improving Medical VQA Consistency with LLM-Augmented Question Sets](https://mlsanity.com/paper/2504.11777/) (2025)
- [Trust in Vision-Language Models: Insights from a Participatory User Workshop](https://mlsanity.com/paper/2511.13458/) (2025)
- [ReasonEdit: Editing Vision-Language Models using Human Reasoning](https://mlsanity.com/paper/2602.02408/) (2026)
- [Bridging the Modality Gap in Roadside LiDAR: A Training-Free Vision-Language Model Framework for Vehicle Classification](https://mlsanity.com/paper/2602.09425/) (2026)
- [Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Tasks](https://mlsanity.com/paper/2603.11689/) (2026)
- [Towards Annotation-Free Validation of MLLMs: A Vision-Language Logical Consistency Metric](https://mlsanity.com/paper/2605.06201/) (2026)
- [Visual Semantic Entropy: Do Vision Language Models Recognize Visual Ambiguity?](https://mlsanity.com/paper/2606.31407/) (2026)
- [When Does Consensus Mean Correctness? Measuring the Agreement-Accuracy Coupling with Semantics-Preserving Re-Rendering](https://mlsanity.com/paper/2608.05670/) (2026)
- [Consistency Has a Computable Blind Spot: A Commutation Theory of Label-Free Reliability for Vision-Language Figure Reading](https://mlsanity.com/paper/2608.05675/) (2026)
- [Target-Checked Reliability Score Refinement for Video Question Answering](https://mlsanity.com/paper/2609.13288/) (2026)
- [Symmetry-Aware Likelihood-Orbit Aggregation for Selective Left-Right Claim Verification](https://mlsanity.com/paper/2609.17004/) (2026)
- [DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs](https://mlsanity.com/paper/2609.28570/) (2026)

## Abstract (arXiv)

The goal of selective prediction is to allow an a model to abstain when it may not be able to deliver a reliable prediction, which is important in safety-critical contexts. Existing approaches to selective prediction typically require access to the internals of a model, require retraining a model or study only unimodal models. However, the most powerful models (e.g. GPT-4) are typically only available as black boxes with inaccessible internals, are not retrainable by end-users, and are frequently used for multimodal tasks. We study the possibility of selective prediction for vision-language models in a realistic, black-box setting. We propose using the principle of textitneighborhood consistency to identify unreliable responses from a black-box vision-language model in question answering tasks. We hypothesize that given only a visual question and model response, the consistency of the model's responses over the neighborhood of a visual question will indicate reliability. It is impossible to directly sample neighbors in feature space in a black-box setting. Instead, we show that it is possible to use a smaller proxy model to approximately sample from the neighborhood. We find that neighborhood consistency can be used to identify model responses to visual questions that are likely unreliable, even in adversarial settings or settings that are out-of-distribution to the proxy model.