mlsanityData to 7 Oct 2026

Multi-View Document Representation Learning for Open-Domain Dense Retrieval

arXiv 2203.08372 ↗First posted 16 Mar 2022cs.CL cs.IRcited by 41

Not on the map. Why?

Abstract (arXiv)

Dense retrieval has achieved impressive advances in first-stage retrieval from a large-scale document collection, which is built on bi-encoder architecture to produce single vector representation of query and document. However, a document can usually answer multiple potential queries from different views. So the single vector representation of a document is hard to match with multi-view queries, and faces a semantic mismatch problem. This paper proposes a multi-view document representation learning framework, aiming to produce multi-view embeddings to represent documents and enforce them to align with different queries. First, we propose a simple yet effective method of generating multiple embeddings through viewers. Second, to prevent multi-view embeddings from collapsing to the same one, we further propose a global-local loss with annealed temperature to encourage the multiple viewers to better align with different potential queries. Experiments show our method outperforms recent works and achieves state-of-the-art results.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 34 of its 42 references that are in the map. First posted on arXiv 16 Mar 2022.

Builds on

34 of 42 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  7. 2017Billion-scale similarity search with GPUscited by 2,477
  8. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  9. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  10. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  11. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  12. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
22 more
  1. 2019Passage Re-ranking with BERTcited by 629
  2. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  3. 2019Revealing the Dark Secrets of BERTcited by 300
  4. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  5. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 213
  6. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  7. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  8. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 164
  9. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  10. 2021Optimizing Dense Retrieval Model Training with Hard Negativescited by 154
  11. 2020Pre-training Tasks for Embedding-based Large-scale Retrievalcited by 153
  12. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  13. 2019Deeper Text Understanding for IR with Contextual Neural Language Modelingcited by 111
  14. 2021Adversarial Retriever-Ranker for dense text retrievalcited by 82
  15. 2021End-to-End Training of Neural Retrievers for Open-Domain Question Answeringcited by 75
  16. 2021Domain-matched Pre-training Tasks for Dense Retrievalcited by 53
  17. 2021PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrievalcited by 50
  18. 2018Approximate Nearest Neighbor Search in High Dimensionscited by 45
  19. 2020Example-Driven Intent Prediction with Observerscited by 26
  20. 2021More Robust Dense Retrieval with Contrastive Dual Learningcited by 15
  21. 2020DiPair: Fast and Accurate Distillation for Trillion-Scale Text Matching and Pair Modelingcited by 14
  22. 2021Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrievalcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 15; 2024: 7; 2025: 8; 2026: 4.

41 papers.

Sort:

By area:

  1. 2022SciRepEval: A Multi-Format Benchmark for Scientific Document Representationscited by 124
  2. 2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88
  3. 2023LLatrieval: LLM-Verified Retrieval for Verifiable Generationcited by 52
  4. 2025MMDocIR: Benchmarking Multimodal Retrieval for Long Documentscited by 43
  5. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  6. 2022Fine-Grained Distillation for Long Document Retrievalcited by 29
  7. 2023Black-box Adversarial Attacks against Dense Retrieval Models: A Multi-view Contrastive Learning Methodcited by 29
  8. 2023Rethinking the Role of Token Retrieval in Multi-Vector Retrievalcited by 28
  9. 2023Dodo: Dynamic Contextual Compression for Decoder-only LMscited by 23
  10. 2022CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrievalcited by 19
  11. 2023Nugget: Neural Agglomerative Embeddings of Textcited by 19
  12. 2023Sub-Sentence Encoder: Contrastive Learning of Propositional Semantic Representationscited by 19
29 more
  1. 2022Mixed-modality Representation Learning and Pre-training for Joint Table-and-Text Retrieval in OpenQAcited by 17
  2. 2023Multiview Identifiers Enhanced Generative Retrievalcited by 16
  3. 2024Question-Based Retrieval using Atomic Units for Enterprise RAGcited by 16
  4. 2024Dense Passage Retrieval: Is it Retrieving?cited by 12
  5. 2023BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrievalcited by 11
  6. 2024Generative Dense Retrieval: Memory Can Be a Burdencited by 11
  7. 2024D2LLM: Decomposed and Distilled Large Language Models for Semantic Searchcited by 9
  8. 2022SeDR: Segment Representation Learning for Long Documents Dense Retrievalcited by 6
  9. 2023Noisy Pair Corrector for Dense Retrievalcited by 5
  10. 2022Learning Diverse Document Representations with Deep Query Interactions for Dense Retrievalcited by 4
  11. 2022Retrieval Oriented Masking Pre-training Language Model for Dense Passage Retrievalcited by 4
  12. 2023PEFA: Parameter-Free Adapters for Large-scale Embedding-based Retrieval Modelscited by 4
  13. 2023Mitigating the Impact of False Negatives in Dense Retrieval with Contrastive Confidence Regularizationcited by 4
  14. 2024Multi-Intent Attribute-Aware Text Matching in Searchingcited by 4
  15. 2024A Survey on Large Language Models from Concept to Implementationcited by 4
  16. 2026Taxonomy of the Retrieval System Framework: Pitfalls and Paradigmscited by 4
  17. 2023Towards Better Entity Linking with Multi-View Enhanced Distillationcited by 3
  18. 2023Enhancing Low-Resource Relation Representations through Multi-View Decouplingcited by 3
  19. 2025Towards Lossless Token Pruning in Late-Interaction Retrieval Modelscited by 2
  20. 2025Dense Retrievers Can Fail on Simple Queries: Revealing The Granularity Dilemma of Embeddingscited by 2
  21. 2025KG-CQR: Leveraging Structured Relation Representations in Knowledge Graphs for Contextual Query Retrievalcited by 1
  22. 2025SemCSE-Multi: Multifaceted and Decodable Embeddings for Aspect-Specific and Interpretable Scientific Domain Mappingcited by 1
  23. 2023QR-CLIP: Introducing Explicit Open-World Knowledge for Location and Time Reasoningcited by 0
  24. 2024Domain Adaptation of Multilingual Semantic Search -- Literature Reviewcited by 0
  25. 2025CroPS: Improving Dense Retrieval with Cross-Perspective Positive Samples in Short-Video Searchcited by 0
  26. 2025brat: Aligned Multi-View Embeddings for Brain MRI Analysiscited by 0
  27. 2026DocRetriever: A Plug-and-Play Framework for Multimodal Document Retrieval with Comprehensive Benchmarkcited by 0
  28. 2026ttda704 at SemEval-2026 Task 4: Modeling Narrative Structures via Pseudonymization and Multi-View Sentence Alignmentcited by 0
  29. 2026Do Large Language Models Capture the Diversity in their Training Data?cited by 0