Explicit Abstention Knobs for Predictable Reliability in Video Question Answering
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
High-stakes deployment of vision-language models (VLMs) requires selective prediction, where systems abstain when uncertain rather than risk costly errors. We investigate whether confidence-based abstention provides reliable control over error rates in video question answering, and whether that control remains robust under distribution shift. Using NExT-QA and Gemini 2.0 Flash, we establish two findings. First, confidence thresholding provides mechanistic control in-distribution. Sweeping threshold epsilon produces smooth risk-coverage tradeoffs, reducing error rates f
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 22 of its 32 references that are in the map. First posted on arXiv 31 Dec 2025.
Builds on
22 of 32 references.
References · 22
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
- 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
- 2017Selective Classification for Deep Neural Networkscited by 799
- 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 735
- 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
- 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
- 2022Conformal Risk Controlcited by 290
10 more
- 2020Selective Question Answering under Domain Shiftcited by 228
- 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
- 2022Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectlycited by 86
- 2021Optimal strategies for reject option classifierscited by 61
- 2023Towards Calibrated Robust Fine-Tuning of Vision-Language Modelscited by 35
- 2022Calibrated Selective Classificationcited by 30
- 2024Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Modelscited by 17
- 2024Selective Classification Under Distribution Shiftscited by 12
- 2020Right Decisions from Wrong Predictions: A Mechanism Design Alternative to Individual Calibrationcited by 7
- 2025Know When to Abstain: Optimal Selective Classification with Likelihood Ratioscited by 5
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Vision-Language Model Confidence Is Not a Property of the Answer
- 2025Data-Driven Calibration of Prediction Sets in Large Vision-Language Models Based on Inductive Conformal Prediction
- 2026Evidence-Order Calibration for Selective Visual Reasoning under Progressive Loss of Question-Critical Evidence
- 2026ReQuest: Rethinking-based Question-Aware Frame Selection for Long-Form Video QA
- 2024Selective "Selective Prediction": Reducing Unnecessary Abstention in Vision-Language Reasoning
- 2026Knowing When Not to Answer: Evaluating Abstention in Multimodal Reasoning Systems
- 2026From Knowing to Abstaining: Bridging the Representation-Action Gap in Vision-Language Models
- 2025Confidence-guided Refinement Reasoning for Zero-shot Question Answering