# Text Editing as Imitation Game

arXiv: https://arxiv.org/abs/2210.12276 · HTML: https://mlsanity.com/paper/2210.12276/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 42 references that are in the map.
- First posted on arXiv 21 Oct 2022.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [Levenshtein Transformer](https://mlsanity.com/paper/1905.11006/) (2019)
- [Encode, Tag, Realize: High-Precision Text Editing](https://mlsanity.com/paper/1909.01187/) (2019)
- [Parallel Iterative Edit Models for Local Sequence Transduction](https://mlsanity.com/paper/1910.02893/) (2019)
- [EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing](https://mlsanity.com/paper/1906.08104/) (2019)
- [Felix: Flexible Text Editing Through Tagging and Insertion](https://mlsanity.com/paper/2003.10687/) (2020)
- [Seq2Edits: Sequence Transduction Using Span-level Edit Operations](https://mlsanity.com/paper/2009.11136/) (2020)
- [Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Task](https://mlsanity.com/paper/1804.05940/) (2018)
- [Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Data](https://mlsanity.com/paper/1903.00138/) (2019)
- [From Language to Programs: Bridging Reinforcement Learning and Maximum Marginal Likelihood](https://mlsanity.com/paper/1704.07926/) (2017)
- [LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2105.08206/) (2021)
- [NL-EDIT: Correcting semantic parse errors through natural language interaction](https://mlsanity.com/paper/2103.14540/) (2021)
- [Multi-granularity Textual Adversarial Attack with Behavior Cloning](https://mlsanity.com/paper/2109.04367/) (2021)
- [Text Generation with Text-Editing Models](https://mlsanity.com/paper/2206.07043/) (2022)
- [An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models](https://mlsanity.com/paper/2203.09486/) (2022)
- [Learning Structural Edits via Incremental Tree Transformations](https://mlsanity.com/paper/2101.12087/) (2021)
- [Sequence-to-Action: Grammatical Error Correction with Action Guided Sequence Generation](https://mlsanity.com/paper/2205.10884/) (2022)
- [Copy that! Editing Sequences by Copying Spans](https://mlsanity.com/paper/2006.04771/) (2020)
- [Incorporating External POS Tagger for Punctuation Restoration](https://mlsanity.com/paper/2106.06731/) (2021)
- [Q-learning with Language Model for Edit-based Unsupervised Summarization](https://mlsanity.com/paper/2010.04379/) (2020)
- [Recurrent Inference in Text Editing](https://mlsanity.com/paper/2009.12643/) (2020)

## Built on by (most cited first, 1 of 1)

- [Extending Multilingual Machine Translation through Imitation Learning](https://mlsanity.com/paper/2311.08538/) (2023)

## Abstract (arXiv)

Text editing, such as grammatical error correction, arises naturally from imperfect textual data. Recent works frame text editing as a multi-round sequence tagging task, where operations -- such as insertion and substitution -- are represented as a sequence of tags. While achieving good results, this encoding is limited in flexibility as all actions are bound to token-level tags. In this work, we reformulate text editing as an imitation game using behavioral cloning. Specifically, we convert conventional sequence-to-sequence data into state-to-action demonstrations, where the action space can be as flexible as needed. Instead of generating the actions one at a time, we introduce a dual decoders structure to parallel the decoding while retaining the dependencies between action tokens, coupled with trajectory augmentation to alleviate the distribution shift that imitation learning often suffers. In experiments on a suite of Arithmetic Equation benchmarks, our model consistently outperforms the autoregressive baselines in terms of performance, efficiency, and robustness. We hope our findings will shed light on future studies in reinforcement learning applying sequence-level action generation to natural language processing.