mlsanityData to 7 Oct 2026

Sentence-aware Contrastive Learning for Open-Domain Passage Retrieval

arXiv 2110.07524 ↗First posted 14 Oct 2021cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Training dense passage representations via contrastive learning has been shown effective for Open-Domain Passage Retrieval (ODPR). Existing studies focus on further optimizing by improving negative sampling strategy or extra pretraining. However, these studies keep unknown in capturing passage with internal representation conflicts from improper modeling granularity. This work thus presents a refined model on the basis of a smaller granularity, contextual sentences, to alleviate the concerned conflicts. In detail, we introduce an in-passage negative sampling strategy to encourage a diverse generation of sentence representations within the same passage. Experiments on three benchmark datasets verify the efficacy of our method, especially on datasets where conflicts are severe. Extensive experiments further present good transferability of our method across datasets.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 14 Oct 2021.

Builds on

27 of 38 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2020Longformer: The Long-Document Transformercited by 3,362
  8. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  9. 2017Billion-scale similarity search with GPUscited by 2,477
  10. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  12. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
15 more
  1. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  2. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  3. 2017Sampling Matters in Deep Embedding Learningcited by 285
  4. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 249
  5. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  6. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  7. 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
  8. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  9. 2020Learning Dense Representations of Phrases at Scalecited by 91
  10. 2021End-to-End Training of Neural Retrievers for Open-Domain Question Answeringcited by 75
  11. 2020SLM: Learning a Discourse Language Representation with Sentence Unshufflingcited by 24
  12. 2020Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentationcited by 18
  13. 2021Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrievalcited by 14
  14. 2021Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategycited by 13
  15. 2020Neural Passage Retrieval with Improved Negative Contrastcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 4; 2024: 1; 2025: 1; 2026: 2.

11 papers.

  1. 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 211
  2. 2022SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrievalcited by 21
  3. 2023BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrievalcited by 11
  4. 2023Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answeringcited by 11
  5. 2022Global and Local Hierarchy-aware Contrastive Framework for Implicit Discourse Relation Recognitioncited by 7
  6. 2022MASTER: Multi-task Pre-trained Bottlenecked Masked Autoencoders are Better Dense Retrieverscited by 7
  7. 2026GENFIG1: Visual Summaries of Scholarly Work as a Challenge for Vision-Language Modelscited by 2
  8. 2023Sentence Representations via Gaussian Embeddingcited by 1
  9. 2024M3: A Multi-Task Mixed-Objective Learning Framework for Open-Domain Multi-Hop Dense Sentence Retrievalcited by 1
  10. 2025Rhapsody: A Dataset for Highlight Detection in Podcastscited by 0
  11. 2026Temporal Preference Optimization for Unsupervised Retrievalcited by 0