# Parallel Iterative Edit Models for Local Sequence Transduction

arXiv: https://arxiv.org/abs/1910.02893 · HTML: https://mlsanity.com/paper/1910.02893/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2019.

## Built on by (most cited first, 30 of 81)

- [A Simple Recipe for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2106.03830/) (2021)
- [PEER: A Collaborative Language Model](https://mlsanity.com/paper/2208.11663/) (2022)
- [SummIt: Iterative Text Summarization via ChatGPT](https://mlsanity.com/paper/2305.14835/) (2023)
- [Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding](https://mlsanity.com/paper/2106.04970/) (2021)
- [Grammatical Error Correction: A Survey of the State of the Art](https://mlsanity.com/paper/2211.05166/) (2022)
- [On Improving Summarization Factual Consistency from Natural Language Feedback](https://mlsanity.com/paper/2212.09968/) (2022)
- [EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Start](https://mlsanity.com/paper/2205.12209/) (2022)
- [LM-Critic: Language Models for Unsupervised Grammatical Error Correction](https://mlsanity.com/paper/2109.06822/) (2021)
- [MuCGEC: a Multi-Reference Multi-Source Evaluation Dataset for Chinese Grammatical Error Correction](https://mlsanity.com/paper/2204.10994/) (2022)
- [SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parser](https://mlsanity.com/paper/2210.12484/) (2022)
- [Text Simplification by Tagging](https://mlsanity.com/paper/2103.05070/) (2021)
- [ReasonBERT: Pre-trained to Reason with Distant Supervision](https://mlsanity.com/paper/2109.04912/) (2021)
- [Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigation](https://mlsanity.com/paper/2305.14734/) (2023)
- [Text Generation with Text-Editing Models](https://mlsanity.com/paper/2206.07043/) (2022)
- [An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models](https://mlsanity.com/paper/2203.09486/) (2022)
- [Improving Iterative Text Revision by Learning Where to Edit from Other Revision Tasks](https://mlsanity.com/paper/2212.01350/) (2022)
- [Enhancing Grammatical Error Correction Systems with Explanations](https://mlsanity.com/paper/2305.15676/) (2023)
- [Generating Bug-Fixes Using Pretrained Transformers](https://mlsanity.com/paper/2104.07896/) (2021)
- [Lossless Acceleration for Seq2seq Generation with Aggressive Decoding](https://mlsanity.com/paper/2205.10350/) (2022)
- [Non-autoregressive Generative Models for Reranking Recommendation](https://mlsanity.com/paper/2402.06871/) (2024)
- [Sequence-to-Action: Grammatical Error Correction with Action Guided Sequence Generation](https://mlsanity.com/paper/2205.10884/) (2022)
- [Disentangled Phonetic Representation for Chinese Spelling Correction](https://mlsanity.com/paper/2305.14783/) (2023)
- [FCGEC: Fine-Grained Corpus for Chinese Grammatical Error Correction](https://mlsanity.com/paper/2210.12364/) (2022)
- [Ensembling and Knowledge Distilling of Large Sequence Taggers for Grammatical Error Correction](https://mlsanity.com/paper/2203.13064/) (2022)
- [Beyond English: Evaluating LLMs for Arabic Grammatical Error Correction](https://mlsanity.com/paper/2312.08400/) (2023)
- [Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction](https://mlsanity.com/paper/2105.04443/) (2021)
- [Lights, Camera, Action! A Framework to Improve NLP Accuracy over OCR documents](https://mlsanity.com/paper/2108.02899/) (2021)
- [Type-Driven Multi-Turn Corrections for Grammatical Error Correction](https://mlsanity.com/paper/2203.09136/) (2022)
- [NaSGEC: a Multi-Domain Chinese Grammatical Error Correction Dataset from Native Speaker Texts](https://mlsanity.com/paper/2305.16023/) (2023)
- [Enhancing Text Editing for Grammatical Error Correction: Arabic as a Case Study](https://mlsanity.com/paper/2503.00985/) (2025)

## Abstract (arXiv)

We present a Parallel Iterative Edit (PIE) model for the problem of local sequence transduction arising in tasks like Grammatical error correction (GEC). Recent approaches are based on the popular encoder-decoder (ED) model for sequence to sequence learning. The ED model auto-regressively captures full dependency among output tokens but is slow due to sequential decoding. The PIE model does parallel decoding, giving up the advantage of modelling full dependency in the output, yet it achieves accuracy competitive with the ED model for four reasons: 1.~predicting edits instead of tokens, 2.~labeling sequences instead of generating sequences, 3.~iteratively refining predictions to capture dependencies, and 4.~factorizing logits over edits and their token argument to harness pre-trained language models like BERT. Experiments on tasks spanning GEC, OCR correction and spell correction demonstrate that the PIE model is an accurate and significantly faster alternative for local sequence transduction.