# Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations

arXiv: https://arxiv.org/abs/2109.13087 · HTML: https://mlsanity.com/paper/2109.13087/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 24 Sep 2021.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)](https://mlsanity.com/paper/1405.5869/) (2014)
- [Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-based Chatbots](https://mlsanity.com/paper/1612.01627/) (2016)
- [Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots](https://mlsanity.com/paper/2004.03588/) (2020)
- [Dialogue Response Ranking Training with Large-Scale Human Feedback Data](https://mlsanity.com/paper/2009.06978/) (2020)
- [Modeling Multi-turn Conversation with Deep Utterance Aggregation](https://mlsanity.com/paper/1806.09102/) (2018)
- [Distilling Dense Representations for Ranking using Tightly-Coupled Teachers](https://mlsanity.com/paper/2010.11386/) (2020)
- [Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Model](https://mlsanity.com/paper/1810.12836/) (2018)
- [Topic-Aware Multi-turn Dialogue Modeling](https://mlsanity.com/paper/2009.12539/) (2020)
- [Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection](https://mlsanity.com/paper/2009.04703/) (2020)
- [Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues](https://mlsanity.com/paper/2009.06265/) (2020)
- [Is Retriever Merely an Approximator of Reader?](https://mlsanity.com/paper/2010.10999/) (2020)
- [OTTers: One-turn Topic Transitions for Open-Domain Dialogue](https://mlsanity.com/paper/2105.13710/) (2021)
- [Deep Retrieval-Based Dialogue Systems: A Short Review](https://mlsanity.com/paper/1907.12878/) (2019)
- [An Effective Domain Adaptive Post-Training Method for BERT in Response Selection](https://mlsanity.com/paper/1908.04812/) (2019)
- [Ultra-Fast, Low-Storage, Highly Effective Coarse-grained Selection in Retrieval-based Chatbot by Using Deep Semantic Hashing](https://mlsanity.com/paper/2012.09647/) (2020)

## Built on by (most cited first, 9 of 9)

- [DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultation](https://mlsanity.com/paper/2308.14346/) (2023)
- [DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuning](https://mlsanity.com/paper/2310.15205/) (2023)
- [DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation](https://mlsanity.com/paper/2204.13031/) (2022)
- [Exploring Dense Retrieval for Dialogue Response Selection](https://mlsanity.com/paper/2110.06612/) (2021)
- [KNSE: A Knowledge-aware Natural Language Inference Framework for Dialogue Symptom Status Recognition](https://mlsanity.com/paper/2305.16833/) (2023)
- [CauESC: A Causal Aware Model for Emotional Support Conversation](https://mlsanity.com/paper/2401.17755/) (2024)
- [Pneg: Prompt-based Negative Response Generation for Dialogue Response Selection Task](https://mlsanity.com/paper/2210.17238/) (2022)
- [K-ESConv: Knowledge Injection for Emotional Support Dialogue Systems via Prompt Learning](https://mlsanity.com/paper/2312.10371/) (2023)
- [On the Effectiveness of Integration Methods for Multimodal Dialogue Response Retrieval](https://mlsanity.com/paper/2506.11499/) (2025)

## Abstract (arXiv)

We study the problem of coarse-grained response selection in retrieval-based dialogue systems. The problem is equally important with fine-grained response selection, but is less explored in existing literature. In this paper, we propose a Contextual Fine-to-Coarse (CFC) distilled model for coarse-grained response selection in open-domain conversations. In our CFC model, dense representations of query, candidate response and corresponding context is learned based on the multi-tower architecture, and more expressive knowledge learned from the one-tower architecture (fine-grained) is distilled into the multi-tower architecture (coarse-grained) to enhance the performance of the retriever. To evaluate the performance of our proposed model, we construct two new datasets based on the Reddit comments dump and Twitter corpus. Extensive experimental results on the two datasets show that the proposed methods achieve a significant improvement over all evaluation metrics compared with traditional baseline methods.