mlsanityData to 7 Oct 2026

Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline

arXiv 2101.08751 ↗First posted 21 Jan 2021cs.IRcited by 65

Not on the map. Why?

Abstract (arXiv)

Pre-trained deep language models~(LM) have advanced the state-of-the-art of text retrieval. Rerankers fine-tuned from deep LM estimates candidate relevance based on rich contextualized matching signals. Meanwhile, deep LMs can also be leveraged to improve search index, building retrievers with better recall. One would expect a straightforward combination of both in a pipeline to have additive performance gain. In this paper, we discover otherwise and that popular reranker cannot fully exploit the improved retrieval result. We, therefore, propose a Localized Contrastive Estimation (LCE) for training rerankers and demonstrate it significantly improves deep two-stage models.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Jan 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 14; 2023: 17; 2024: 14; 2025: 14; 2026: 2.

65 papers.

Sort:

By area:

  1. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  2. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 233
  3. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  4. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  5. 2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 114
  6. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  7. 2023Zero-Shot Listwise Document Reranking with a Large Language Modelcited by 81
  8. 2023RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Modelscited by 69
  9. 2022Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generationcited by 47
  10. 2024JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Rerankingcited by 34
  11. 2022Fine-Grained Distillation for Long Document Retrievalcited by 29
  12. 2023WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpuscited by 29
38 more
  1. 2022Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrievalcited by 26
  2. 2022In Defense of Cross-Encoders for Zero-Shot Retrievalcited by 21
  3. 2022HLATR: Enhance Multi-stage Text Retrieval with Hybrid List Aware Transformer Rerankingcited by 19
  4. 2025Silent Leaks: Implicit Knowledge Extraction Attack on RAG Systems through Benign Queriescited by 19
  5. 2024Multi-Head RAG: Solving Multi-Aspect Problems with LLMscited by 18
  6. 2024Metasql: A Generate-then-Rank Framework for Natural Language to SQL Translationcited by 17
  7. 2023DataFinder: Scientific Dataset Recommendation from Natural Language Descriptionscited by 15
  8. 2023Wikiformer: Pre-training with Structured Information of Wikipedia for Ad-hoc Retrievalcited by 15
  9. 2024RankMamba: Benchmarking Mamba's Document Ranking Performance in the Era of Transformerscited by 15
  10. 2023Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Modelscited by 14
  11. 2023Transforming Human-Centered AI Collaboration: Redefining Embodied Agents Capabilities through Interactive Grounded Language Instructionscited by 13
  12. 2023MGeo: Multi-Modal Geographic Pre-Training Methodcited by 12
  13. 2023THUIR2 at NTCIR-16 Session Search (SS) Taskcited by 12
  14. 2022Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classificationcited by 10
  15. 2024STARD: A Chinese Statute Retrieval Dataset with Real Queries Issued by Non-professionalscited by 10
  16. 2024State Space Models are Strong Text Rerankerscited by 10
  17. 2022Long Document Re-ranking with Modular Re-rankercited by 9
  18. 2022Neural Rankers for Effective Screening Prioritisation in Medical Systematic Review Literature Searchcited by 9
  19. 2023A Lightweight Constrained Generation Alternative for Query-focused Summarizationcited by 8
  20. 2024Drowning in Documents: Consequences of Scaling Reranker Inferencecited by 8
  21. 2024Self-Calibrated Listwise Reranking with Large Language Modelscited by 7
  22. 2022HYRR: Hybrid Infused Reranking for Passage Retrievalcited by 6
  23. 2023Bio-SIEVE: Exploring Instruction Tuning Large Language Models for Systematic Review Automationcited by 6
  24. 2023Generating Natural Language Queries for More Effective Systematic Review Screening Prioritisationcited by 6
  25. 2025Distillation versus Contrastive Learning: How to Train Your Rerankerscited by 6
  26. 2025ERank: Fusing Supervised Fine-Tuning and Reinforcement Learning for Effective and Efficient Text Rerankingcited by 6
  27. 2025Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generationcited by 5
  28. 2025CoRank: LLM-Based Compact Reranking with Document Features for Scientific Retrievalcited by 5
  29. 2025Universal Biological Sequence Reranking for Improved De Novo Peptide Sequencingcited by 5
  30. 2023Relevance Feedback with Brain Signalscited by 4
  31. 2022Do Lessons from Metric Learning Generalize to Image-Caption Retrieval?cited by 3
  32. 2024A Bi-metric Framework for Fast Similarity Searchcited by 3
  33. 2024Prompts as Auto-Optimized Training Hyperparameters: Training Best-in-Class IR Models from Scratch with 10 Gold Labelscited by 3
  34. 2024Entity Insertion in Multilingual Linked Corpora: The Case of Wikipediacited by 3
  35. 2025ListConRanker: A Contrastive Text Reranker with Listwise Encodingcited by 3
  36. 2022Pre-training for Information Retrieval: Are Hyperlinks Fully Explored?cited by 2
  37. 2023ExaRanker: Explanation-Augmented Neural Rankercited by 2
  38. 2025Can Cross Encoders Produce Useful Sentence Embeddings?cited by 2

50 most cited of 65 All 65 (JSON)