Towards Trustworthy Reranking: A Simple yet Effective Abstention Mechanism
Not on the map. Why?
Abstract (arXiv)
Neural Information Retrieval (NIR) has significantly improved upon heuristic-based Information Retrieval (IR) systems. Yet, failures remain frequent, the models used often being unable to retrieve documents relevant to the user's query. We address this challenge by proposing a lightweight abstention mechanism tailored for real-world constraints, with particular emphasis placed on the reranking phase. We introduce a protocol for evaluating abstention strategies in black-box scenarios (typically encountered when relying on API services), demonstrating their efficacy, and propose a simple yet effective data-driven mechanism. We provide open-source code for experiment replication and abstention implementation, fostering wider adoption and application in diverse contexts.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 48 of its 98 references that are in the map. First posted on arXiv 20 Feb 2024.
Builds on
48 of 98 references.
References · 48
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
- 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
- 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
- 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
- 2020Energy-based Out-of-distribution Detectioncited by 1,296
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
36 more
- 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
- 2017Deep Bayesian Active Learning with Image Datacited by 812
- 2017Selective Classification for Deep Neural Networkscited by 799
- 2019Passage Re-ranking with BERTcited by 629
- 2022MTEB: Massive Text Embedding Benchmarkcited by 623
- 2021Datasets: A Community Library for Natural Language Processingcited by 526
- 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
- 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 426
- 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
- 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
- 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
- 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
- 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
- 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
- 2020Selective Question Answering under Domain Shiftcited by 228
- 2021A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detectioncited by 206
- 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
- 2022Dense Text Retrieval based on Pretrained Language Models: A Surveycited by 136
- 2017A Deep Relevance Matching Model for Ad-hoc Retrievalcited by 126
- 2019A General Framework for Uncertainty Estimation in Deep Learningcited by 119
- 2023Learning to Filter Context for Retrieval-Augmented Generationcited by 82
- 2021Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detectioncited by 67
- 2020The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizescited by 48
- 2021Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learningcited by 42
- 2018Confidence Modeling for Neural Semantic Parsingcited by 33
- 2017Neural Models for Information Retrievalcited by 29
- 2015Semi-supervised Question Retrieval with Gated Convolutionscited by 28
- 2021A statistical framework for efficient out of distribution detection in deep neural networkscited by 27
- 2023Ranking with Abstentioncited by 27
- 2022Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Datacited by 21
- 2023Knowledge-Augmented Language Model Verificationcited by 21
- 2022Igeood: An Information Geometry Approach to Out-of-Distribution Detectioncited by 20
- 2021A Pseudo-Metric between Probability Distributions based on Depth-Trimmed Regionscited by 17
- 2023Unsupervised Layer-wise Score Aggregation for Textual OOD Detectioncited by 15
- 2023Alloprof: a new French question-answer education dataset and its use in an information retrieval case studycited by 6
- 2023A Novel Information-Theoretic Objective to Disentangle Representations for Fair Classificationcited by 1
Built on by
Citing papers per year: 2024: 1.
1 paper.