# More Robust Dense Retrieval with Contrastive Dual Learning

arXiv: https://arxiv.org/abs/2107.07773 · HTML: https://mlsanity.com/paper/2107.07773/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2021.

## Built on by (most cited first, 15 of 15)

- [Retrieval-Enhanced Machine Learning](https://mlsanity.com/paper/2205.01230/) (2022)
- [Multi-View Document Representation Learning for Open-Domain Dense Retrieval](https://mlsanity.com/paper/2203.08372/) (2022)
- [ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance](https://mlsanity.com/paper/2502.17057/) (2025)
- [Typo-Robust Representation Learning for Dense Retrieval](https://mlsanity.com/paper/2306.10348/) (2023)
- [Building A Coding Assistant via the Retrieval-Augmented Language Model](https://mlsanity.com/paper/2410.16229/) (2024)
- [Through the Stealth Lens: Attention-Aware Defenses Against Poisoning in RAG](https://mlsanity.com/paper/2506.04390/) (2025)
- [IMRNNs: An Efficient Method for Interpretable Dense Retrieval via Embedding Modulation](https://mlsanity.com/paper/2601.20084/) (2026)
- [Prototypical Extreme Multi-label Classification with a Dynamic Margin Loss](https://mlsanity.com/paper/2410.20401/) (2024)
- [Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Models](https://mlsanity.com/paper/2502.13656/) (2025)
- [RATE: Reviewer Profiling and Annotation-free Training for Expertise Ranking in Peer Review Systems](https://mlsanity.com/paper/2601.19637/) (2026)
- [From RAG to QA-RAG: Integrating Generative AI for Pharmaceutical Regulatory Compliance Process](https://mlsanity.com/paper/2402.01717/) (2024)
- [Query-oriented Data Augmentation for Session Search](https://mlsanity.com/paper/2407.03720/) (2024)
- [SEAL: Structure and Element Aware Learning to Improve Long Structured Document Retrieval](https://mlsanity.com/paper/2508.20778/) (2025)
- [MERIT: Matching Expertise via Rubric-Informed Training for Reviewer Assignment](https://mlsanity.com/paper/2605.27865/) (2026)
- [When Does Dense Retrieval Need Asymmetric Geometry? A Bias-Variance Theory of Shared and Dual Projections](https://mlsanity.com/paper/2609.32488/) (2026)

## Abstract (arXiv)

Dense retrieval conducts text retrieval in the embedding space and has shown many advantages compared to sparse retrieval. Existing dense retrievers optimize representations of queries and documents with contrastive training and map them to the embedding space. The embedding space is optimized by aligning the matched query-document pairs and pushing the negative documents away from the query. However, in such training paradigm, the queries are only optimized to align to the documents and are coarsely positioned, leading to an anisotropic query embedding space. In this paper, we analyze the embedding space distributions and propose an effective training paradigm, Contrastive Dual Learning for Approximate Nearest Neighbor (DANCE) to learn fine-grained query representations for dense retrieval. DANCE incorporates an additional dual training object of query retrieval, inspired by the classic information retrieval training axiom, query likelihood. With contrastive learning, the dual training object of DANCE learns more tailored representations for queries and documents to keep the embedding space smooth and uniform, thriving on the ranking performance of DANCE on the MS MARCO document retrieval task. Different from ANCE that only optimized with the document retrieval task, DANCE concentrates the query embeddings closer to document representations while making the document distribution more discriminative. Such concentrated query embedding distribution assigns more uniform negative sampling probabilities to queries and helps to sufficiently optimize query representations in the query retrieval task. Our codes are released at https://github.com/thunlp/DANCE.