# Encode, Tag, Realize: High-Precision Text Editing

arXiv: https://arxiv.org/abs/1909.01187 · HTML: https://mlsanity.com/paper/1909.01187/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Sep 2019.

## Built on by (most cited first, 30 of 99)

- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation](https://mlsanity.com/paper/2203.16487/) (2022)
- [A Simple Recipe for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2106.03830/) (2021)
- [A Design Space for Intelligent and Interactive Writing Assistants](https://mlsanity.com/paper/2403.14117/) (2024)
- [Understanding Iterative Revision from Human-Written Text](https://mlsanity.com/paper/2203.03802/) (2022)
- [Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding](https://mlsanity.com/paper/2106.04970/) (2021)
- [Grammatical Error Correction: A Survey of the State of the Art](https://mlsanity.com/paper/2211.05166/) (2022)
- [DiffusER: Discrete Diffusion via Edit-based Reconstruction](https://mlsanity.com/paper/2210.16886/) (2022)
- [LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2105.08206/) (2021)
- [On Improving Summarization Factual Consistency from Natural Language Feedback](https://mlsanity.com/paper/2212.09968/) (2022)
- [EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Start](https://mlsanity.com/paper/2205.12209/) (2022)
- [ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews](https://mlsanity.com/paper/2306.12587/) (2023)
- [NL-EDIT: Correcting semantic parse errors through natural language interaction](https://mlsanity.com/paper/2103.14540/) (2021)
- [MuCGEC: a Multi-Reference Multi-Source Evaluation Dataset for Chinese Grammatical Error Correction](https://mlsanity.com/paper/2204.10994/) (2022)
- [Focus Attention: Promoting Faithfulness and Diversity in Summarization](https://mlsanity.com/paper/2105.11921/) (2021)
- [Learning to Model Editing Processes](https://mlsanity.com/paper/2205.12374/) (2022)
- [EditEval: An Instruction-Based Benchmark for Text Improvements](https://mlsanity.com/paper/2209.13331/) (2022)
- [Text Simplification by Tagging](https://mlsanity.com/paper/2103.05070/) (2021)
- [Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits](https://mlsanity.com/paper/2301.00355/) (2023)
- [Linguistic Rules-Based Corpus Generation for Native Chinese Grammatical Error Correction](https://mlsanity.com/paper/2210.10442/) (2022)
- [Learning to Complete Code with Sketches](https://mlsanity.com/paper/2106.10158/) (2021)
- [Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigation](https://mlsanity.com/paper/2305.14734/) (2023)
- [Text Generation with Text-Editing Models](https://mlsanity.com/paper/2206.07043/) (2022)
- [An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models](https://mlsanity.com/paper/2203.09486/) (2022)
- [Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks](https://mlsanity.com/paper/2212.01350/) (2022)
- [Idiomatic Expression Identification using Semantic Compatibility](https://mlsanity.com/paper/2110.10064/) (2021)
- [GPT-3.5 for Grammatical Error Correction](https://mlsanity.com/paper/2405.08469/) (2024)
- [Text Revision by On-the-Fly Representation Optimization](https://mlsanity.com/paper/2204.07359/) (2022)
- [Learning Structural Edits via Incremental Tree Transformations](https://mlsanity.com/paper/2101.12087/) (2021)
- [Med-EASi: Finely Annotated Dataset and Models for Controllable Simplification of Medical Texts](https://mlsanity.com/paper/2302.09155/) (2023)

## Abstract (arXiv)

We propose LaserTagger - a sequence tagging approach that casts text generation as a text editing task. Target texts are reconstructed from the inputs using three main edit operations: keeping a token, deleting it, and adding a phrase before the token. To predict the edit operations, we propose a novel model, which combines a BERT encoder with an autoregressive Transformer decoder. This approach is evaluated on English text on four tasks: sentence fusion, sentence splitting, abstractive summarization, and grammar correction. LaserTagger achieves new state-of-the-art results on three of these tasks, performs comparably to a set of strong seq2seq baselines with a large number of training examples, and outperforms them when the number of examples is limited. Furthermore, we show that at inference time tagging can be more than two orders of magnitude faster than comparable seq2seq models, making it more attractive for running in a live environment.