# Few-Shot Conversational Dense Retrieval

arXiv: https://arxiv.org/abs/2105.04166 · HTML: https://mlsanity.com/paper/2105.04166/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2021.

## Built on by (most cited first, 30 of 72)

- [Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewriting](https://mlsanity.com/paper/2310.09716/) (2023)
- [ConvGQR: Generative Query Reformulation for Conversational Search](https://mlsanity.com/paper/2305.15645/) (2023)
- [Dialog Inpainting: Turning Documents into Dialogs](https://mlsanity.com/paper/2205.09073/) (2022)
- [MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents](https://mlsanity.com/paper/2109.12595/) (2021)
- [Neural Approaches to Conversational Information Retrieval](https://mlsanity.com/paper/2201.05176/) (2022)
- [UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems](https://mlsanity.com/paper/2401.13256/) (2024)
- [Fine-Grained Distillation for Long Document Retrieval](https://mlsanity.com/paper/2212.10423/) (2022)
- [CHIQ: Contextual History Enhancement for Improving Query Rewriting in Conversational Search](https://mlsanity.com/paper/2406.05013/) (2024)
- [Learning to Relate to Previous Turns in Conversational Search](https://mlsanity.com/paper/2306.02553/) (2023)
- [Ask Optimal Questions: Aligning Large Language Models with Retriever's Preference in Conversation](https://mlsanity.com/paper/2402.11827/) (2024)
- [A Survey of Conversational Search](https://mlsanity.com/paper/2410.15576/) (2024)
- [Saving Dense Retriever from Shortcut Dependency in Conversational Search](https://mlsanity.com/paper/2202.07280/) (2022)
- [ChatRetriever: Adapting Large Language Models for Generalized and Robust Conversational Dense Retrieval](https://mlsanity.com/paper/2404.13556/) (2024)
- [Query Performance Prediction using Relevance Judgments Generated by Large Language Models](https://mlsanity.com/paper/2404.01012/) (2024)
- [Generalizing Conversational Dense Retrieval via LLM-Cognition Data Augmentation](https://mlsanity.com/paper/2402.07092/) (2024)
- [ConTextual Masked Auto-Encoder for Dense Passage Retrieval](https://mlsanity.com/paper/2208.07670/) (2022)
- [Query Performance Prediction: From Ad-hoc to Conversational Search](https://mlsanity.com/paper/2305.10923/) (2023)
- [Adaptive Query Rewriting: Aligning Rewriters through Marginal Probability of Conversational Answers](https://mlsanity.com/paper/2406.10991/) (2024)
- [Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Survey](https://mlsanity.com/paper/2208.03197/) (2022)
- [AdaCQR: Enhancing Query Reformulation for Conversational Search via Sparse and Dense Retrieval Alignment](https://mlsanity.com/paper/2407.01965/) (2024)
- [AgentIR: Reasoning-Aware Retrieval for Deep Research Agents](https://mlsanity.com/paper/2603.04384/) (2026)
- [UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations](https://mlsanity.com/paper/2507.07030/) (2025)
- [Exploring Dense Retrieval for Dialogue Response Selection](https://mlsanity.com/paper/2110.06612/) (2021)
- [TPE: Towards Better Compositional Reasoning over Conceptual Tools with Multi-persona Collaboration](https://mlsanity.com/paper/2309.16090/) (2023)
- [How to Leverage Personal Textual Knowledge for Personalized Conversational Information Retrieval](https://mlsanity.com/paper/2407.16192/) (2024)
- [Aligning Query Representation with Rewritten Query and Relevance Judgments in Conversational Search](https://mlsanity.com/paper/2407.20189/) (2024)
- [Talk the Walk: Synthetic Data Generation for Conversational Music Recommendation](https://mlsanity.com/paper/2301.11489/) (2023)
- [DH-RAG: A Dynamic Historical Context-Powered Retrieval-Augmented Generation Method for Multi-Turn Dialogue](https://mlsanity.com/paper/2502.13847/) (2025)
- [ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance](https://mlsanity.com/paper/2502.17057/) (2025)
- [Query-as-context Pre-training for Dense Passage Retrieval](https://mlsanity.com/paper/2212.09598/) (2022)

## Abstract (arXiv)

Dense retrieval (DR) has the potential to resolve the query understanding challenge in conversational search by matching in the learned embedding space. However, this adaptation is challenging due to DR models' extra needs for supervision signals and the long-tail nature of conversational search. In this paper, we present a Conversational Dense Retrieval system, ConvDR, that learns contextualized embeddings for multi-turn conversational queries and retrieves documents solely using embedding dot products. In addition, we grant ConvDR few-shot ability using a teacher-student framework, where we employ an ad hoc dense retriever as the teacher, inherit its document encodings, and learn a student query encoder to mimic the teacher embeddings on oracle reformulated queries. Our experiments on TREC CAsT and OR-QuAC demonstrate ConvDR's effectiveness in both few-shot and fully-supervised settings. It outperforms previous systems that operate in the sparse word space, matches the retrieval accuracy of oracle query reformulations, and is also more efficient thanks to its simplicity. Our analyses reveal that the advantages of ConvDR come from its ability to capture informative context while ignoring the unrelated context in previous conversation rounds. This makes ConvDR more effective as conversations evolve while previous systems may get confused by the increased noise from previous turns. Our code is publicly available at https://github.com/thunlp/ConvDR.