mlsanityData to 7 Oct 2026

Questions Are All You Need to Train a Dense Passage Retriever

arXiv 2206.10658 ↗First posted 21 Jun 2022cs.CL cs.IRcited by 47

Not on the map. Why?

Abstract (arXiv)

We introduce ART, a new corpus-level autoencoding approach for training dense retrieval models that does not require any labeled training data. Dense retrieval is a central challenge for open-domain tasks, such as Open QA, where state-of-the-art methods typically require large supervised datasets with custom hard-negative mining and denoising of positive examples. ART, in contrast, only requires access to unpaired inputs and outputs (e.g. questions and potential answer documents). It uses a new document-retrieval autoencoding scheme, where (1) an input question is used to retrieve a set of evidence documents, and (2) the documents are then used to compute the probability of reconstructing the original question. Training for retrieval based on question reconstruction enables effective unsupervised learning of both document and question encoders, which can be later incorporated into complete Open QA systems without any further finetuning. Extensive experiments demonstrate that ART obtains state-of-the-art results on multiple QA retrieval benchmarks with only generic initialization from a pre-trained language model, removing the need for labeled data and task-specific losses.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 43 of its 52 references that are in the map. First posted on arXiv 21 Jun 2022.

Builds on

43 of 52 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  11. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
31 more
  1. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  2. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  4. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  5. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  6. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  7. 2019Passage Re-ranking with BERTcited by 629
  8. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  9. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  10. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  11. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  12. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
  13. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  14. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 213
  15. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  16. 2020Generation-Augmented Retrieval for Open-domain Question Answeringcited by 186
  17. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  18. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 164
  19. 2022Improving Passage Retrieval with Zero-Shot Question Generationcited by 164
  20. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  21. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  22. 2019Learning Dense Representations for Entity Retrievalcited by 122
  23. 2020XOR QA: Cross-lingual Open-Retrieval Question Answeringcited by 121
  24. 2021Adversarial Retriever-Ranker for dense text retrievalcited by 82
  25. 2021End-to-End Training of Neural Retrievers for Open-Domain Question Answeringcited by 75
  26. 2020Relevance-guided Supervision for OpenQA with ColBERTcited by 71
  27. 2022InPars: Data Augmentation for Information Retrieval using Large Language Modelscited by 60
  28. 2022Dialog Inpainting: Turning Documents into Dialogscited by 56
  29. 2021Learning to Retrieve Passages without Supervisioncited by 46
  30. 2020Beyond [CLS] through Ranking by Generationcited by 20
  31. 2022Hyperlink-induced Pre-training for Passage Retrieval in Open-domain Question Answeringcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 15; 2024: 17; 2025: 8; 2026: 2.

47 papers.

Sort:

By area:

  1. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  2. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  3. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  4. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  5. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  6. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  7. 2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
  8. 2022Task-aware Retrieval with Instructionscited by 107
  9. 2023Improving Language Models via Plug-and-Play Retrieval Feedbackcited by 87
  10. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 81
  11. 2023RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Modelscited by 69
  12. 2023LLatrieval: LLM-Verified Retrieval for Verifiable Generationcited by 52
35 more
  1. 2024GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Modelscited by 42
  2. 2023Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Trainingcited by 31
  3. 2023Rethinking the Role of Token Retrieval in Multi-Vector Retrievalcited by 28
  4. 2023IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositionscited by 25
  5. 2022Visconde: Multi-document QA with GPT-3 and Neural Rerankingcited by 23
  6. 2023Discrete Prompt Optimization via Constrained Generation for Zero-shot Re-rankercited by 20
  7. 2023Instruction Distillation Makes Large Language Models Efficient Zero-shot Rankerscited by 19
  8. 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspectivecited by 15
  9. 2022Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Surveycited by 14
  10. 2025LLM4Ranking: An Easy-to-use Framework of Utilizing Large Language Models for Document Rerankingcited by 12
  11. 2023Bring Your Own KG: Self-Supervised Program Synthesis for Zero-Shot KGQAcited by 11
  12. 2024Harnessing Multi-Role Capabilities of Large Language Models for Open-Domain Question Answeringcited by 8
  13. 2024Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Modelscited by 7
  14. 2024Inference Scaling for Bridging Retrieval and Augmented Generationcited by 7
  15. 2023Exploring the Practicality of Generative Retrieval on Dynamic Corporacited by 6
  16. 2023Noisy Pair Corrector for Dense Retrievalcited by 5
  17. 2022AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentationcited by 3
  18. 2024Unsupervised Multilingual Dense Retrieval via Generative Pseudo Labelingcited by 3
  19. 2024Disentangling Questions from Query Generation for Task-Adaptive Retrievalcited by 3
  20. 2024Zero-Shot Dense Retrieval with Embeddings from Relevance Feedbackcited by 3
  21. 2024Best Practices for Distilling Large Language Models into BERT for Web Search Rankingcited by 3
  22. 2024Large Language Model as a Teacher for Zero-shot Tagging at Extreme Scalescited by 2
  23. 2024A Statistical Framework for Data-dependent Retrieval-Augmented Modelscited by 2
  24. 2025Gumbel Reranking: Differentiable End-to-End Reranker Optimizationcited by 2
  25. 2025Optimizing open-domain question answering with graph-based retrieval augmented generationcited by 2
  26. 2025Synergistic Weak-Strong Collaboration by Aligning Preferencescited by 2
  27. 2025PersonalAI: A Systematic Comparison of Knowledge Graph Storage and Retrieval Approaches for Personalized LLM agentscited by 2
  28. 2024ENCO: Life-Cycle Management of Enterprise-Grade Copilotscited by 1
  29. 2024SKETCH: Structured Knowledge Enhanced Text Comprehension for Holistic Retrievalcited by 1
  30. 2025InSQuAD: In-Context Learning for Efficient Retrieval via Submodular Mutual Information to Enforce Quality and Diversitycited by 1
  31. 2024On the Necessity of World Knowledge for Mitigating Missing Labels in Extreme Classificationcited by 0
  32. 2025The Other Side of the Coin: Exploring Fairness in Retrieval-Augmented Generationcited by 0
  33. 2025Enhancing Transformer-Based Rerankers with Synthetic Data and LLM-Based Supervisioncited by 0
  34. 2026ReToP: Learning to Rewrite Electronic Health Records for Clinical Predictioncited by 0
  35. 2026Structural Rationale Distillation via Reasoning Space Compressioncited by 0