# Self-Attention for Incomplete Utterance Rewriting

arXiv: https://arxiv.org/abs/2202.12160 · HTML: https://mlsanity.com/paper/2202.12160/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [Improving Multi-turn Dialogue Modelling with Utterance ReWriter](https://mlsanity.com/paper/1906.07004/) (2019)
- [Incomplete Utterance Rewriting as Semantic Segmentation](https://mlsanity.com/paper/2009.13166/) (2020)
- [SARG: A Novel Semi Autoregressive Generator for Multi-turn Incomplete Utterance Restoration](https://mlsanity.com/paper/2008.01474/) (2020)
- [Semantic Role Labeling Guided Multi-turn Dialogue ReWriter](https://mlsanity.com/paper/2010.01417/) (2020)
- [Robust Dialogue Utterance Rewriting as Sequence Tagging](https://mlsanity.com/paper/2012.14535/) (2020)

## Built on by (most cited first, 2 of 2)

- [Incomplete Utterance Rewriting as Sequential Greedy Tagging](https://mlsanity.com/paper/2307.06337/) (2023)
- [RSMLP: A light Sampled MLP Structure for Incomplete Utterance Rewrite](https://mlsanity.com/paper/2502.12587/) (2025)

## Abstract (arXiv)

Incomplete utterance rewriting (IUR) has recently become an essential task in NLP, aiming to complement the incomplete utterance with sufficient context information for comprehension. In this paper, we propose a novel method by directly extracting the coreference and omission relationship from the self-attention weight matrix of the transformer instead of word embeddings and edit the original text accordingly to generate the complete utterance. Benefiting from the rich information in the self-attention weight matrix, our method achieved competitive results on public IUR datasets.