Certified Error Control of Candidate Set Pruning for Two-Stage Relevance Ranking
Not on the map. Why?
Abstract (arXiv)
In information retrieval (IR), candidate set pruning has been commonly used to speed up two-stage relevance ranking. However, such an approach lacks accurate error control and often trades accuracy off against computational efficiency in an empirical fashion, lacking theoretical guarantees. In this paper, we propose the concept of certified error control of candidate set pruning for relevance ranking, which means that the test error after pruning is guaranteed to be controlled under a user-specified threshold with high probability. Both in-domain and out-of-domain experiments show that our method successfully prunes the first-stage retrieved candidate sets to improve the second-stage reranking speed while satisfying the pre-specified accuracy constraints in both settings. For example, on MS MARCO Passage v1, our method yields an average candidate set size of 27 out of 1,000 which increases the reranking speed by about 37 times, while the MRR@10 is greater than a pre-specified value of 0.38 with about 90% empirical coverage and the empirical baselines fail to provide such guarantee. Code and data are available at: https://github.com/alexlimh/CEC-Ranking.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 52 references that are in the map. First posted on arXiv 19 May 2022.
Builds on
24 of 52 references.
References · 24
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2017Billion-scale similarity search with GPUscited by 2,497
- 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
- 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
- 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
- 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
- 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
- 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
- 2021Distribution-Free, Risk-Controlling Prediction Setscited by 288
- 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 286
- 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 226
- 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 214
12 more
- 2019Multi-Stage Document Ranking with BERTcited by 201
- 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
- 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
- 2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
- 2021Learning Passage Impacts for Inverted Indexescited by 68
- 2019PAC Confidence Sets for Deep Neural Networks via Calibrated Predictioncited by 60
- 2020Calibrated Prediction with Covariate Shift via Unsupervised Domain Adaptationcited by 43
- 2021A Proposed Conceptual Framework for a Representational Approach to Information Retrievalcited by 24
- 2020PAC Confidence Predictions for Deep Neural Network Classifierscited by 21
- 2020The Cascade Transformer: an Application for Efficient Answer Sentence Selectioncited by 20
- 2020Query-level Early Exit for Additive Learning-to-Rank Ensemblescited by 1
- 2021Learning Early Exit Strategies for Additive Ranking Ensemblescited by 1
Built on by
Citing papers per year: 2024: 1.
1 paper.