# Sentence-aware Contrastive Learning for Open-Domain Passage Retrieval

arXiv: https://arxiv.org/abs/2110.07524 · HTML: https://mlsanity.com/paper/2110.07524/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 14 Oct 2021.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2010.08191/) (2020)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://mlsanity.com/paper/2105.11741/) (2021)
- [Distilling Knowledge from Reader to Retriever for Question Answering](https://mlsanity.com/paper/2012.04584/) (2020)
- [TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agents](https://mlsanity.com/paper/1901.08149/) (2019)
- [CLEAR: Contrastive Learning for Sentence Representation](https://mlsanity.com/paper/2012.15466/) (2020)
- [Learning Dense Representations of Phrases at Scale](https://mlsanity.com/paper/2012.12624/) (2020)
- [End-to-End Training of Neural Retrievers for Open-Domain Question Answering](https://mlsanity.com/paper/2101.00408/) (2021)
- [SLM: Learning a Discourse Language Representation with Sentence Unshuffling](https://mlsanity.com/paper/2010.16249/) (2020)
- [Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentation](https://mlsanity.com/paper/2009.13815/) (2020)
- [Improving Document Representations by Generating Pseudo Query Embeddings for Dense Retrieval](https://mlsanity.com/paper/2105.03599/) (2021)
- [Graph-free Multi-hop Reading Comprehension: A Select-to-Guide Strategy](https://mlsanity.com/paper/2107.11823/) (2021)
- [Neural Passage Retrieval with Improved Negative Contrast](https://mlsanity.com/paper/2010.12523/) (2020)

## Built on by (most cited first, 11 of 11)

- [Query Rewriting for Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2305.14283/) (2023)
- [SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval](https://mlsanity.com/paper/2210.11773/) (2022)
- [BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrieval](https://mlsanity.com/paper/2305.11052/) (2023)
- [Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answering](https://mlsanity.com/paper/2309.02233/) (2023)
- [Global and Local Hierarchy-aware Contrastive Framework for Implicit Discourse Relation Recognition](https://mlsanity.com/paper/2211.13873/) (2022)
- [MASTER: Multi-task Pre-trained Bottlenecked Masked Autoencoders are Better Dense Retrievers](https://mlsanity.com/paper/2212.07841/) (2022)
- [GENFIG1: Visual Summaries of Scholarly Work as a Challenge for Vision-Language Models](https://mlsanity.com/paper/2604.04172/) (2026)
- [Sentence Representations via Gaussian Embedding](https://mlsanity.com/paper/2305.12990/) (2023)
- [M3: A Multi-Task Mixed-Objective Learning Framework for Open-Domain Multi-Hop Dense Sentence Retrieval](https://mlsanity.com/paper/2403.14074/) (2024)
- [Rhapsody: A Dataset for Highlight Detection in Podcasts](https://mlsanity.com/paper/2505.19429/) (2025)
- [Temporal Preference Optimization for Unsupervised Retrieval](https://mlsanity.com/paper/2606.17664/) (2026)

## Abstract (arXiv)

Training dense passage representations via contrastive learning has been shown effective for Open-Domain Passage Retrieval (ODPR). Existing studies focus on further optimizing by improving negative sampling strategy or extra pretraining. However, these studies keep unknown in capturing passage with internal representation conflicts from improper modeling granularity. This work thus presents a refined model on the basis of a smaller granularity, contextual sentences, to alleviate the concerned conflicts. In detail, we introduce an in-passage negative sampling strategy to encourage a diverse generation of sentence representations within the same passage. Experiments on three benchmark datasets verify the efficacy of our method, especially on datasets where conflicts are severe. Extensive experiments further present good transferability of our method across datasets.