# Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QA

arXiv: https://arxiv.org/abs/2305.01812 · HTML: https://mlsanity.com/paper/2305.01812/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 56 references that are in the map.
- First posted on arXiv 2 May 2023.

## Builds on (most cited first, 30 of 41)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://mlsanity.com/paper/1903.00161/) (2019)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Universal Adversarial Triggers for Attacking and Analyzing NLP](https://mlsanity.com/paper/1908.07125/) (2019)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020)
- [PAWS: Paraphrase Adversaries from Word Scrambling](https://mlsanity.com/paper/1904.01130/) (2019)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [Synthetic and Natural Noise Both Break Neural Machine Translation](https://mlsanity.com/paper/1711.02173/) (2017)
- [Adversarial Example Generation with Syntactically Controlled Paraphrase Networks](https://mlsanity.com/paper/1804.06059/) (2018)
- [Contrastive Test-Time Adaptation](https://mlsanity.com/paper/2204.10377/) (2022)
- [Selective Question Answering under Domain Shift](https://mlsanity.com/paper/2006.09462/) (2020)
- [SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine](https://mlsanity.com/paper/1704.05179/) (2017)
- [MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension](https://mlsanity.com/paper/1910.09753/) (2019)
- [Open-Domain Question Answering Goes Conversational via Question Rewriting](https://mlsanity.com/paper/2010.04898/) (2020)
- [Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information](https://mlsanity.com/paper/1805.04655/) (2018)
- [The Effect of Natural Distribution Shift on Question Answering Models](https://mlsanity.com/paper/2004.14444/) (2020)
- [DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension](https://mlsanity.com/paper/1804.07927/) (2018)
- [Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settings](https://mlsanity.com/paper/2203.00211/) (2022)
- [Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions](https://mlsanity.com/paper/2109.05794/) (2021)
- [ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ)](https://mlsanity.com/paper/2009.11352/) (2020)

## Built on by (most cited first, 20 of 20)

- [Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models](https://mlsanity.com/paper/2305.19187/) (2023)
- [A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation](https://mlsanity.com/paper/2307.03987/) (2023)
- [Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Models](https://mlsanity.com/paper/2407.16221/) (2024)
- [Mitigating Temporal Misalignment by Discarding Outdated Facts](https://mlsanity.com/paper/2305.14824/) (2023)
- [Trustworthy and Practical AI for Healthcare: A Guided Deferral System with Large Language Models](https://mlsanity.com/paper/2406.07212/) (2024)
- [Knowing When to Abstain: Medical LLMs Under Clinical Uncertainty](https://mlsanity.com/paper/2601.12471/) (2026)
- [Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness](https://mlsanity.com/paper/2407.01942/) (2024)
- [Crowd-Calibrator: Can Annotator Disagreement Inform Calibration in Subjective Tasks?](https://mlsanity.com/paper/2408.14141/) (2024)
- [LLMs' Reading Comprehension Is Affected by Parametric Knowledge and Struggles with Hypothetical Statements](https://mlsanity.com/paper/2404.06283/) (2024)
- [CausalAbstain: Enhancing Multilingual LLMs with Causal Reasoning for Trustworthy Abstention](https://mlsanity.com/paper/2506.00519/) (2025)
- [Ambiguity Meets Uncertainty: Investigating Uncertainty Estimation for Word Sense Disambiguation](https://mlsanity.com/paper/2305.13119/) (2023)
- [Efficiently Deploying LLMs with Controlled Risk](https://mlsanity.com/paper/2410.02173/) (2024)
- [Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitation](https://mlsanity.com/paper/2511.11500/) (2025)
- [Can NLP Models 'Identify', 'Distinguish', and 'Justify' Questions that Don't have a Definitive Answer?](https://mlsanity.com/paper/2309.04635/) (2023)
- [Aligning Language Models with Selective Prediction](https://mlsanity.com/paper/2607.03528/) (2026)
- [A Unified Evaluation Framework for Novelty Detection and Accommodation in NLP with an Instantiation in Authorship Attribution](https://mlsanity.com/paper/2305.05079/) (2023)
- [PassiveQA: A Three-Action Framework for Epistemically Calibrated Question Answering via Supervised Finetuning](https://mlsanity.com/paper/2604.04565/) (2026)
- [SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring](https://mlsanity.com/paper/2604.25855/) (2026)
- [NoisyCoconut: Counterfactual Consensus via Latent Space Reasoning](https://mlsanity.com/paper/2605.08221/) (2026)
- [From Knowing to Abstaining: Bridging the Representation-Action Gap in Vision-Language Models](https://mlsanity.com/paper/2609.32653/) (2026)

## Abstract (arXiv)

Despite remarkable progress made in natural language processing, even the state-of-the-art models often make incorrect predictions. Such predictions hamper the reliability of systems and limit their widespread adoption in real-world applications. 'Selective prediction' partly addresses the above concern by enabling models to abstain from answering when their predictions are likely to be incorrect. While selective prediction is advantageous, it leaves us with a pertinent question 'what to do after abstention'. To this end, we present an explorative study on 'Post-Abstention', a task that allows re-attempting the abstained instances with the aim of increasing 'coverage' of the system without significantly sacrificing its 'accuracy'. We first provide mathematical formulation of this task and then explore several methods to solve it. Comprehensive experiments on 11 QA datasets show that these methods lead to considerable risk improvements -- performance metric of the Post-Abstention task -- both in the in-domain and the out-of-domain settings. We also conduct a thorough analysis of these results which further leads to several interesting findings. Finally, we believe that our work will encourage and facilitate further research in this important area of addressing the reliability of NLP systems.