mlsanityData to 7 Oct 2026

Boosted Dense Retriever

arXiv 2112.07771 ↗First posted 14 Dec 2021cs.CL cs.IRcited by 10

Not on the map. Why?

Abstract (arXiv)

We propose DrBoost, a dense retrieval ensemble inspired by boosting. DrBoost is trained in stages: each component model is learned sequentially and specialized by focusing only on retrieval mistakes made by the current ensemble. The final representation is the concatenation of the output vectors of all the component models, making it a drop-in replacement for standard dense retrievers at test time. DrBoost enjoys several advantages compared to standard dense retrieval models. It produces representations which are 4x more compact, while delivering comparable retrieval results. It also performs surprisingly well under approximate search with coarse quantization, reducing latency and bandwidth needs by another 4x. In practice, this can make the difference between serving indices from disk versus from memory, paving the way for much cheaper deployments.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 43 of its 84 references that are in the map. First posted on arXiv 14 Dec 2021.

Builds on

43 of 84 references.

References · 43

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2017Billion-scale similarity search with GPUscited by 2,477
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  8. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  9. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  10. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  11. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  12. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
31 more
  1. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  2. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  3. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  4. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  5. 2020Autoregressive Entity Retrievalcited by 333
  6. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
  7. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  8. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  9. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 213
  10. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 198
  11. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 164
  12. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  13. 2021Optimizing Dense Retrieval Model Training with Hard Negativescited by 154
  14. 2020Pre-training Tasks for Embedding-based Large-scale Retrievalcited by 153
  15. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 143
  16. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  17. 2021Adversarial Retriever-Ranker for dense text retrievalcited by 82
  18. 2021End-to-End Training of Neural Retrievers for Open-Domain Question Answeringcited by 75
  19. 2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 59
  20. 2021Efficient Passage Retrieval with Hashing for Open-domain Question Answeringcited by 53
  21. 2021Domain-matched Pre-training Tasks for Dense Retrievalcited by 53
  22. 2021NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learnedcited by 52
  23. 2021Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performancecited by 35
  24. 2020A Memory Efficient Baseline for Open Domain Question Answeringcited by 32
  25. 2020Is Retriever Merely an Approximator of Reader?cited by 30
  26. 2021Learning Discrete Representations via Constrained Clustering for Effective and Efficient Dense Retrievalcited by 27
  27. 2020Understanding BERT Rankers Under Distillationcited by 15
  28. 2019Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embeddingcited by 12
  29. 2021Language-agnostic Topic Classification for Wikipediacited by 6
  30. 2021Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answeringcited by 5
  31. 2020BoostingBERT:Integrating Multi-Class Boosting into BERT for NLP Taskscited by 3

Built on by

Cited by year

Citing papers per year: 2022: 9; 2023: 0; 2024: 1.

10 papers.

  1. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 112
  2. 2022Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reductioncited by 21
  3. 2022A Survey for Efficient Open Domain Question Answeringcited by 20
  4. 2022A Study on the Efficiency and Generalization of Light Hybrid Retrieverscited by 7
  5. 2022Entity-Focused Dense Passage Retrieval for Outside-Knowledge Visual Question Answeringcited by 7
  6. 2022Optimizing Test-Time Query Representations for Dense Retrievalcited by 6
  7. 2022Query Expansion Using Contextual Clue Sampling with Language Modelscited by 5
  8. 2022Reduce Catastrophic Forgetting of Dense Retrieval Training with Teleportation Negativescited by 5
  9. 2022Injecting Domain Adaptation with Learning-to-hash for Effective and Efficient Zero-shot Dense Retrievalcited by 4
  10. 2024More Room for Language: Investigating the Effect of Retrieval on Language Modelscited by 2