mlsanityData to 7 Oct 2026

Levenshtein Transformer

arXiv 1905.11006 ↗First posted 27 May 2019cs.CL cs.LGcited by 197

Not on the map. Why?

Cited by 36 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Modern neural sequence generation models are built to either generate tokens step-by-step from scratch or (iteratively) modify a sequence of tokens bounded by a fixed length. In this work, we develop Levenshtein Transformer, a new partially autoregressive model devised for more flexible and amenable sequence generation. Unlike previous approaches, the atomic operations of our model are insertion and deletion. The combination of them facilitates not only generation but also sequence refinement allowing dynamic length changes. We also propose a set of new training techniques dedicated at them, effectively exploiting one as the other's learning signal thanks to their complementary nature. Experiments applying the proposed model achieve comparable performance but much-improved efficiency on both generation (e.g. machine translation, text summarization) and refinement tasks (e.g. automatic post-editing). We further confirm the flexibility of our model by showing a Levenshtein Transformer trained by machine translation can straightforwardly be used for automatic post-editing.

Key facts. Cited by 197 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 56; 2022: 56; 2023: 35; 2024: 17; 2025: 11; 2026: 22.

197 papers.

Sort:

By area:

  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  2. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  3. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 385
  4. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
  5. 2022Continuous diffusion for categorical datacited by 202
  6. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
  7. 2023Speculative Decoding with Big Little Decodercited by 163
  8. 2023A Reparameterized Discrete Diffusion Model for Text Generationcited by 152
  9. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  10. 2021Learning to Efficiently Sample from Diffusion Probabilistic Modelscited by 129
  11. 2024Randomized Autoregressive Visual Generationcited by 125
  12. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 113
38 more
  1. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  2. 2022MiniViT: Compressing Vision Transformers with Weight Multiplexingcited by 88
  3. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoisecited by 79
  4. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codescited by 71
  5. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  6. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformerscited by 63
  7. 2023TESS: Text-to-Text Self-Conditioned Simplex Diffusioncited by 63
  8. 2021Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translationcited by 57
  9. 2021Instantaneous Grammatical Error Correction with Shallow Aggressive Decodingcited by 53
  10. 2025Edit Flows: Flow Matching with Edit Operationscited by 53
  11. 2023DINOISER: Diffused Conditional Sequence Learning by Manipulating Noisescited by 50
  12. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Textscited by 49
  13. 2022Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 44
  14. 2021FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognitioncited by 43
  15. 2023Diffusion Models for Non-autoregressive Text Generation: A Surveycited by 43
  16. 2022Controllable Text Generation with Neurally-Decomposed Oraclecited by 41
  17. 2022DiffusER: Discrete Diffusion via Edit-based Reconstructioncited by 41
  18. 2021Non-Autoregressive Translation with Layer-Wise Prediction and Deep Supervisioncited by 40
  19. 2022On Improving Summarization Factual Consistency from Natural Language Feedbackcited by 38
  20. 2022EdiT5: Semi-Autoregressive Text-Editing with T5 Warm-Startcited by 35
  21. 2021Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learningcited by 34
  22. 2021Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translationcited by 34
  23. 2022TranSpeech: Speech-to-Speech Translation With Bilateral Perturbationcited by 34
  24. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  25. 2022SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parsercited by 32
  26. 2022Learning to Model Editing Processescited by 30
  27. 2023Learning to Reason and Memorize with Self-Notescited by 30
  28. 2021Text Simplification by Taggingcited by 27
  29. 2021FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognitioncited by 27
  30. 2021A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generationcited by 27
  31. 2022On the Learning of Non-Autoregressive Transformerscited by 27
  32. 2023Second Thoughts are Best: Learning to Re-Align With Human Values from Text Editscited by 27
  33. 2023Iterative Translation Refinement with Large Language Modelscited by 27
  34. 2021Progressive Multi-Granularity Training for Non-Autoregressive Translationcited by 25
  35. 2023Don't lose the message while paraphrasing: A study on content preserving style transfercited by 25
  36. 2021Parallel Refinements for Lexically Constrained Text Generation with BARTcited by 24
  37. 2022Levenshtein OCRcited by 24
  38. 2021Investigating the Reordering Capability in CTC-based Non-Autoregressive End-to-End Speech Translationcited by 23

50 most cited of 197 All 197 (JSON)