# ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation

arXiv: https://arxiv.org/abs/1907.05339 · HTML: https://mlsanity.com/paper/1907.05339/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2019.

## Built on by (most cited first, 30 of 32)

- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Less is More: Learning to Refine Dialogue History for Personalized Dialogue Generation](https://mlsanity.com/paper/2204.08128/) (2022)
- [The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions](https://mlsanity.com/paper/2310.12418/) (2023)
- [One Chatbot Per Person: Creating Personalized Chatbots based on Implicit User Profiles](https://mlsanity.com/paper/2108.09355/) (2021)
- [Multimodal Dialogue Response Generation](https://mlsanity.com/paper/2110.08515/) (2021)
- [ChatGPT as your Personal Data Scientist](https://mlsanity.com/paper/2305.13657/) (2023)
- [Maria: A Visual Experience Powered Conversational Agent](https://mlsanity.com/paper/2105.13073/) (2021)
- [Thinking Clearly, Talking Fast: Concept-Guided Non-Autoregressive Generation for Open-Domain Dialogue Systems](https://mlsanity.com/paper/2109.04084/) (2021)
- [Learning to Select Context in a Hierarchical and Global Perspective for Open-domain Dialogue Generation](https://mlsanity.com/paper/2102.09282/) (2021)
- [Commonsense and Named Entity Aware Knowledge Grounded Dialogue Generation](https://mlsanity.com/paper/2205.13928/) (2022)
- [PopALM: Popularity-Aligned Language Models for Social Media Trendy Response Prediction](https://mlsanity.com/paper/2402.18950/) (2024)
- [MORQA: Benchmarking Evaluation Metrics for Medical Open-Ended Question Answering](https://mlsanity.com/paper/2509.12405/) (2025)
- [MulZDG: Multilingual Code-Switching Framework for Zero-shot Dialogue Generation](https://mlsanity.com/paper/2208.08629/) (2022)
- [CDialog: A Multi-turn Covid-19 Conversation Dataset for Entity-Aware Dialog Generation](https://mlsanity.com/paper/2212.06049/) (2022)
- [MCP: Self-supervised Pre-training for Personalized Chatbots with Multi-level Contrastive Sampling](https://mlsanity.com/paper/2210.08753/) (2022)
- [Towards Efficient Dialogue Pre-training with Transferable and Interpretable Latent Structure](https://mlsanity.com/paper/2210.12461/) (2022)
- [FCM: A Fine-grained Comparison Model for Multi-turn Dialogue Reasoning](https://mlsanity.com/paper/2109.10510/) (2021)
- [Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning](https://mlsanity.com/paper/2110.00159/) (2021)
- [Collaborative Reasoning on Multi-Modal Semantic Graphs for Video-Grounded Dialogue Generation](https://mlsanity.com/paper/2210.12460/) (2022)
- [THINK: A Novel Conversation Model for Generating Grammatically Correct and Coherent Responses](https://mlsanity.com/paper/2105.13630/) (2021)
- [WeaSuL: Weakly Supervised Dialogue Policy Learning: Reward Estimation for Multi-turn Dialogue](https://mlsanity.com/paper/2108.01487/) (2021)
- [A Speaker-aware Parallel Hierarchical Attentive Encoder-Decoder Model for Multi-turn Dialogue Generation](https://mlsanity.com/paper/2110.06823/) (2021)
- [What Did You Say? Task-Oriented Dialog Datasets Are Not Conversational!?](https://mlsanity.com/paper/2203.03431/) (2022)
- [Learning to Express in Knowledge-Grounded Conversation](https://mlsanity.com/paper/2204.05805/) (2022)
- [Stop Filtering: Multi-View Attribute-Enhanced Dialogue Learning](https://mlsanity.com/paper/2205.11206/) (2022)
- [DialoGen: Generalized Long-Range Context Representation for Dialogue Systems](https://mlsanity.com/paper/2210.06282/) (2022)
- [Prompt-Guided Injection of Conformation to Pre-trained Protein Model](https://mlsanity.com/paper/2202.02944/) (2022)
- [A Neural Conversation Generation Model via Equivalent Shared Memory Investigation](https://mlsanity.com/paper/2108.09164/) (2021)
- [X-ReCoSa: Multi-Scale Context Aggregation For Multi-Turn Dialogue Generation](https://mlsanity.com/paper/2303.07833/) (2023)
- [A Static and Dynamic Attention Framework for Multi Turn Dialogue Generation](https://mlsanity.com/paper/2410.20766/) (2024)

## Abstract (arXiv)

In multi-turn dialogue generation, response is usually related with only a few contexts. Therefore, an ideal model should be able to detect these relevant contexts and produce a suitable response accordingly. However, the widely used hierarchical recurrent encoderdecoder models just treat all the contexts indiscriminately, which may hurt the following response generation process. Some researchers try to use the cosine similarity or the traditional attention mechanism to find the relevant contexts, but they suffer from either insufficient relevance assumption or position bias problem. In this paper, we propose a new model, named ReCoSa, to tackle this problem. Firstly, a word level LSTM encoder is conducted to obtain the initial representation of each context. Then, the self-attention mechanism is utilized to update both the context and masked response representation. Finally, the attention weights between each context and response representations are computed and used in the further decoding process. Experimental results on both Chinese customer services dataset and English Ubuntu dialogue dataset show that ReCoSa significantly outperforms baseline models, in terms of both metric-based and human evaluations. Further analysis on attention shows that the detected relevant contexts by ReCoSa are highly coherent with human's understanding, validating the correctness and interpretability of ReCoSa.