# Neural Passage Retrieval with Improved Negative Contrast

arXiv: https://arxiv.org/abs/2010.12523 · HTML: https://mlsanity.com/paper/2010.12523/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2020.

## Built on by (most cited first, 11 of 11)

- [RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking](https://mlsanity.com/paper/2110.07367/) (2021)
- [PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrieval](https://mlsanity.com/paper/2108.06027/) (2021)
- [A Thorough Examination on Zero-shot Dense Retrieval](https://mlsanity.com/paper/2204.12755/) (2022)
- [Mixed-modality Representation Learning and Pre-training for Joint Table-and-Text Retrieval in OpenQA](https://mlsanity.com/paper/2210.05197/) (2022)
- [ConTextual Masked Auto-Encoder for Dense Passage Retrieval](https://mlsanity.com/paper/2208.07670/) (2022)
- [Contrastive Fine-tuning Improves Robustness for Neural Rankers](https://mlsanity.com/paper/2105.12932/) (2021)
- [Sentence-aware Contrastive Learning for Open-Domain Passage Retrieval](https://mlsanity.com/paper/2110.07524/) (2021)
- [Query-as-context Pre-training for Dense Passage Retrieval](https://mlsanity.com/paper/2212.09598/) (2022)
- [Adam: Dense Retrieval Distillation with Adaptive Dark Examples](https://mlsanity.com/paper/2212.10192/) (2022)
- [Towards Graph-hop Retrieval and Reasoning in Complex Question Answering over Textual Database](https://mlsanity.com/paper/2305.14211/) (2023)
- [Domain Adaptation of Multilingual Semantic Search -- Literature Review](https://mlsanity.com/paper/2402.02932/) (2024)

## Abstract (arXiv)

In this paper we explore the effects of negative sampling in dual encoder models used to retrieve passages for automatic question answering. We explore four negative sampling strategies that complement the straightforward random sampling of negatives, typically used to train dual encoder models. Out of the four strategies, three are based on retrieval and one on heuristics. Our retrieval-based strategies are based on the semantic similarity and the lexical overlap between questions and passages. We train the dual encoder models in two stages: pre-training with synthetic data and fine tuning with domain-specific data. We apply negative sampling to both stages. The approach is evaluated in two passage retrieval tasks. Even though it is not evident that there is one single sampling strategy that works best in all the tasks, it is clear that our strategies contribute to improving the contrast between the response and all the other passages. Furthermore, mixing the negatives from different strategies achieve performance on par with the best performing strategy in all tasks. Our results establish a new state-of-the-art level of performance on two of the open-domain question answering datasets that we evaluated.