# CO3: Low-resource Contrastive Co-training for Generative Conversational Query Rewrite

arXiv: https://arxiv.org/abs/2403.11873 · HTML: https://mlsanity.com/paper/2403.11873/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 58 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Question Rewriting for Conversational Question Answering](https://mlsanity.com/paper/2004.14652/) (2020)
- [Few-Shot Generative Conversational Query Rewriting](https://mlsanity.com/paper/2006.05009/) (2020)
- [Few-Shot Conversational Dense Retrieval](https://mlsanity.com/paper/2105.04166/) (2021)
- [TREC CAsT 2019: The Conversational Assistance Track Overview](https://mlsanity.com/paper/2003.13624/) (2020)
- [Improving Multi-turn Dialogue Modelling with Utterance ReWriter](https://mlsanity.com/paper/1906.07004/) (2019)
- [Query Resolution for Conversational Search with Limited Supervision](https://mlsanity.com/paper/2005.11723/) (2020)
- [Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment](https://mlsanity.com/paper/1806.06478/) (2018)
- [Contextualized Query Embeddings for Conversational Search](https://mlsanity.com/paper/2104.08707/) (2021)
- [Incomplete Utterance Rewriting as Semantic Segmentation](https://mlsanity.com/paper/2009.13166/) (2020)
- [Scaling Multi-Domain Dialogue State Tracking via Query Reformulation](https://mlsanity.com/paper/1903.05164/) (2019)
- [Stay Hungry, Stay Focused: Generating Informative and Specific Questions in Information-Seeking Conversations](https://mlsanity.com/paper/2004.14530/) (2020)
- [A Comparison of Question Rewriting Methods for Conversational Passage Retrieval](https://mlsanity.com/paper/2101.07382/) (2021)
- [Reinforced Dynamic Reasoning for Conversational Question Generation](https://mlsanity.com/paper/1907.12667/) (2019)
- [Semantic Role Labeling Guided Multi-turn Dialogue ReWriter](https://mlsanity.com/paper/2010.01417/) (2020)
- [Topic Propagation in Conversational Search](https://mlsanity.com/paper/2004.14054/) (2020)
- [Unsupervised Conversation Disentanglement through Co-Training](https://mlsanity.com/paper/2109.03199/) (2021)
- [Reinforced Co-Training](https://mlsanity.com/paper/1804.06035/) (2018)
- [Filling Conversation Ellipsis for Better Social Dialog Understanding](https://mlsanity.com/paper/1911.10776/) (2019)

## Abstract (arXiv)

Generative query rewrite generates reconstructed query rewrites using the conversation history while rely heavily on gold rewrite pairs that are expensive to obtain. Recently, few-shot learning is gaining increasing popularity for this task, whereas these methods are sensitive to the inherent noise due to limited data size. Besides, both attempts face performance degradation when there exists language style shift between training and testing cases. To this end, we study low-resource generative conversational query rewrite that is robust to both noise and language style shift. The core idea is to utilize massive unlabeled data to make further improvements via a contrastive co-training paradigm. Specifically, we co-train two dual models (namely Rewriter and Simplifier) such that each of them provides extra guidance through pseudo-labeling for enhancing the other in an iterative manner. We also leverage contrastive learning with data augmentation, which enables our model pay more attention on the truly valuable information than the noise. Extensive experiments demonstrate the superiority of our model under both few-shot and zero-shot scenarios. We also verify the better generalization ability of our model when encountering language style shift.