# Conversational Query Rewriting with Self-supervised Learning

arXiv: https://arxiv.org/abs/2102.04708 · HTML: https://mlsanity.com/paper/2102.04708/

## Key facts

- Cited by 5 papers in the map (data through 7 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 9 Feb 2021.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [A Neural Network Approach to Context-Sensitive Generation of Conversational Responses](https://mlsanity.com/paper/1506.06714/) (2015)
- [From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots](https://mlsanity.com/paper/1801.01957/) (2018)
- [Few-Shot Generative Conversational Query Rewriting](https://mlsanity.com/paper/2006.05009/) (2020)
- [Improving Multi-turn Dialogue Modelling with Utterance ReWriter](https://mlsanity.com/paper/1906.07004/) (2019)
- [End-to-End Task-Completion Neural Dialogue Systems](https://mlsanity.com/paper/1703.01008/) (2017)
- [The JDDC Corpus: A Large-Scale Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service](https://mlsanity.com/paper/1911.09969/) (2019)
- [Incomplete Utterance Rewriting as Semantic Segmentation](https://mlsanity.com/paper/2009.13166/) (2020)
- [AliMe Assist: An Intelligent Assistant for Creating an Innovative E-commerce Experience](https://mlsanity.com/paper/1801.05032/) (2018)
- [Filling Conversation Ellipsis for Better Social Dialog Understanding](https://mlsanity.com/paper/1911.10776/) (2019)
- [Sequential Attention-based Network for Noetic End-to-End Response Selection](https://mlsanity.com/paper/1901.02609/) (2019)
- [MLR: A Two-stage Conversational Query Rewriting Model with Multi-task Learning](https://mlsanity.com/paper/2004.05812/) (2020)

## Built on by (most cited first, 5 of 5)

- [Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey](https://mlsanity.com/paper/2503.22458/) (2025)
- [Annotation-Free Reinforcement Learning Query Rewriting via Verifiable Search Reward](https://mlsanity.com/paper/2507.23242/) (2025)
- [Utterance Rewriting with Contrastive Learning in Multi-turn Dialogue](https://mlsanity.com/paper/2203.11587/) (2022)
- [QURG: Question Rewriting Guided Context-Dependent Text-to-SQL Semantic Parsing](https://mlsanity.com/paper/2305.06655/) (2023)
- [RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents](https://mlsanity.com/paper/2605.26352/) (2026)

## Abstract (arXiv)

Context modeling plays a critical role in building multi-turn dialogue systems. Conversational Query Rewriting (CQR) aims to simplify the multi-turn dialogue modeling into a single-turn problem by explicitly rewriting the conversational query into a self-contained utterance. However, existing approaches rely on massive supervised training data, which is labor-intensive to annotate. And the detection of the omitted important information from context can be further improved. Besides, intent consistency constraint between contextual query and rewritten query is also ignored. To tackle these issues, we first propose to construct a large-scale CQR dataset automatically via self-supervised learning, which does not need human annotation. Then we introduce a novel CQR model Teresa based on Transformer, which is enhanced by self-attentive keywords detection and intent consistency constraint. Finally, we conduct extensive experiments on two public datasets. Experimental results demonstrate that our proposed model outperforms existing CQR baselines significantly, and also prove the effectiveness of self-supervised learning on improving the CQR performance.