mlsanityData to 7 Oct 2026

Learning to Model Editing Processes

arXiv 2205.12374 ↗First posted 24 May 2022cs.CL cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

Most existing sequence generation models produce outputs in one pass, usually left-to-right. However, this is in contrast with a more natural approach that humans use in generating content; iterative refinement and editing. Recent work has introduced edit-based models for various tasks (such as neural machine translation and text style transfer), but these generally model a single edit step. In this work, we propose modeling editing processes, modeling the whole process of iteratively generating sequences. We form a conceptual framework to describe the likelihood of multi-step edits, and describe neural models that can learn a generative model of sequences based on these multistep edits. We introduce baseline results and metrics on this task, finding that modeling editing processes improves performance on a variety of axes on both our proposed task and related downstream tasks compared to previous single-step models of edits.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 29 of its 44 references that are in the map. First posted on arXiv 24 May 2022.

Builds on

29 of 44 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  8. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  9. 2017Style Transfer from Non-Parallel Text by Cross-Alignmentcited by 246
  10. 2017End-to-end Neural Coreference Resolutioncited by 240
  11. 2018Adaptive Input Representations for Neural Language Modelingcited by 226
  12. 2019Levenshtein Transformercited by 197
17 more
  1. 2018Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinementcited by 194
  2. 2020GECToR -- Grammatical Error Correction: Tag, Not Rewritecited by 177
  3. 2019Encode, Tag, Realize: High-Precision Text Editingcited by 99
  4. 2018CGMH: Constrained Sentence Generation by Metropolis-Hastings Samplingcited by 93
  5. 2020Felix: Flexible Text Editing Through Tagging and Insertioncited by 58
  6. 2022Understanding Iterative Revision from Human-Written Textcited by 56
  7. 2022Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revisioncited by 46
  8. 2021LEWIS: Levenshtein Editing for Unsupervised Text Style Transfercited by 40
  9. 2020Unsupervised Text Generation by Learning from Searchcited by 36
  10. 2018Learning to Represent Editscited by 34
  11. 2020Text Editing by Commandcited by 33
  12. 2018WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discoursecited by 26
  13. 2017A Neural Architecture for Generating Natural Language Descriptions from Source Code Changescited by 23
  14. 2021Learning Structural Edits via Incremental Tree Transformationscited by 16
  15. 2020Variational Inference for Learning Representations of Natural Language Editscited by 5
  16. 2019An Edit-centric Approach for Wikipedia Article Quality Assessmentcited by 3
  17. 2018Text Morphingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 19; 2024: 6; 2025: 1; 2026: 1.

30 papers.

Sort:

By area:

  1. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  2. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  3. 2022Generating Sequences by Learning to Self-Correctcited by 271
  4. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 178
  5. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 118
  6. 2023RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputscited by 104
  7. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 99
  8. 2024Aligner: Efficient Alignment by Learning to Correctcited by 96
  9. 2022PEER: A Collaborative Language Modelcited by 86
  10. 2023CoEdIT: Text Editing by Task-Specific Instruction Tuningcited by 67
  11. 2024Aligning LLM Agents by Learning Latent Preference from User Editscited by 66
  12. 2023Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generationcited by 55
18 more
  1. 2023SummIt: Iterative Text Summarization via ChatGPTcited by 53
  2. 2024Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Editscited by 49
  3. 2022DiffusER: Discrete Diffusion via Edit-based Reconstructioncited by 41
  4. 2023ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviewscited by 34
  5. 2023Second Thoughts are Best: Learning to Re-Align With Human Values from Text Editscited by 27
  6. 2023Flows: Building Blocks of Reasoning and Collaborating AIcited by 25
  7. 2023Data-Efficient Alignment of Large Language Models with Human Feedback Through Natural Languagecited by 22
  8. 2024The Better Angels of Machine Personality: How Personality Relates to LLM Safetycited by 21
  9. 2023GrACE: Generation using Associated Code Editscited by 16
  10. 2024Artificial Intelligence for Operations Research: Revolutionizing the Operations Research Processcited by 11
  11. 2023Coeditor: Leveraging Contextual Changes for Multi-round Code Auto-editingcited by 9
  12. 2023Non-autoregressive Text Editing with Copy-aware Latent Alignmentscited by 8
  13. 2024Grounding Natural Language to SQL Translation with Data-Based Self-Explanationscited by 7
  14. 2023XATU: A Fine-grained Instruction-based Benchmark for Explainable Text Updatescited by 6
  15. 2023Automatic and Human-AI Interactive Text Generationcited by 4
  16. 2023DECap: Towards Generalized Explicit Caption Editing via Diffusion Mechanismcited by 1
  17. 2025Learning Extrapolative Sequence Transformations from Markov Chainscited by 0
  18. 2026EditSR: Enhancing Neural Symbolic Regression via Edit-based Rectificationcited by 0