mlsanityData to 7 Oct 2026

Non-Monotonic Sequential Text Generation

arXiv 1902.02192 ↗First posted 5 Feb 2019cs.CL cs.LG stat.MLcited by 55

Not on the map. Why?

Abstract (arXiv)

Standard sequential generation methods assume a pre-specified generation order, such as text generation methods which generate words from left to right. In this work, we propose a framework for training models of text generation that operate in non-monotonic orders; the model directly learns good orders, without any additional annotation. Our framework operates by generating a word at an arbitrary position, and then recursively generating words to its left and then words to its right, yielding a binary tree. Learning is framed as imitation learning, including a coaching method which moves from imitating an oracle to reinforcing the policy's own preferences. Experimental results demonstrate that using the proposed method, it is possible to learn policies which generate text without pre-specifying a generation order, while achieving competitive performance with conventional left-to-right generation.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 16; 2022: 11; 2023: 8; 2024: 5; 2025: 6; 2026: 8.

55 papers.

Sort:

By area:

  1. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  2. 2023Speculative Decoding with Big Little Decodercited by 163
  3. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 153
  4. 2022RankGen: Improving Text Generation with Large Ranking Modelscited by 72
  5. 2023Factually Consistent Summarization via Reinforcement Learning with Textual Entailment Feedbackcited by 71
  6. 2020Fighting Copycat Agents in Behavioral Cloning from Observation Historiescited by 70
  7. 2025Generalized Interpolating Discrete Diffusioncited by 68
  8. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  9. 2021Machine Translation Decoding beyond Beam Searchcited by 55
  10. 2022Plansformer: Generating Symbolic Plans using Transformerscited by 41
  11. 2022Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Surveycited by 38
  12. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
38 more
  1. 2021Learning to Complete Code with Sketchescited by 23
  2. 2022An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Modelscited by 20
  3. 2022Why is constrained neural language generation particularly challenging?cited by 19
  4. 2021Learning Structural Edits via Incremental Tree Transformationscited by 16
  5. 2026DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvascited by 16
  6. 2021Discovering Non-monotonic Autoregressive Orderings with Variational Inferencecited by 15
  7. 2021InsNet: An Efficient, Flexible, and Performant Insertion-based Text Generation Modelcited by 14
  8. 2023Zero-shot Clarifying Question Generation for Conversational Searchcited by 14
  9. 2021IGA : An Intent-Guided Authoring Assistantcited by 12
  10. 2023PathFinder: Guided Search over Multi-Step Reasoning Pathscited by 12
  11. 2024The Belief State Transformercited by 12
  12. 2025Insertion Language Models: Sequence Generation with Arbitrary-Position Insertionscited by 11
  13. 2021Exploring Dynamic Selection of Branch Expansion Orders for Code Generationcited by 10
  14. 2024Retrieval is Accurate Generationcited by 10
  15. 2022Can Language Models Be Specific? How?cited by 8
  16. 2021Leveraging Recursive Gumbel-Max Trick for Approximate Inference in Combinatorial Spacescited by 7
  17. 2022Linguistically inspired roadmap for building biologically reliable protein language modelscited by 7
  18. 2023Advancements in Scientific Controllable Text Generation Methodscited by 7
  19. 2022UCEpic: Unifying Aspect Planning and Lexical Constraints for Generating Explanations in Recommendationcited by 6
  20. 2024Improving Grammatical Error Correction via Contextual Data Augmentationcited by 6
  21. 2021Multi-View Feature Representation for Dialogue Generation with Bidirectional Distillationcited by 5
  22. 2025Looking beyond the next tokencited by 5
  23. 2021Data-to-text Generation by Splicing Together Nearest Neighborscited by 4
  24. 2021ENCONTER: Entity Constrained Progressive Sequence Generation via Insertion-based Transformercited by 3
  25. 2021Extending Multi-Sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applicationscited by 3
  26. 2023Self-Infilling Code Generationcited by 3
  27. 2024Untangle the KNOT: Interweaving Conflicting Knowledge and Reasoning Skills in Large Language Modelscited by 3
  28. 2024COrAL: Order-Agnostic Language Modeling for Efficient Iterative Refinementcited by 3
  29. 2021Diformer: Directional Transformer for Neural Machine Translationcited by 2
  30. 2023Interactive Text Generationcited by 2
  31. 2021Improving Scheduled Sampling with Elastic Weight Consolidation for Neural Machine Translationcited by 1
  32. 2021Towards More Efficient Insertion Transformer with Fractional Positional Encodingcited by 1
  33. 2022An Ion Exchange Mechanism Inspired Story Ending Generator for Different Characterscited by 1
  34. 2025What Makes the Preferred Thinking Direction for LLMs in Multiple-choice Questions?cited by 1
  35. 2025Distilling Specialized Orders for Visual Generationcited by 1
  36. 2025Discovering Learning-Friendly Generation Orders for Sequential Computationcited by 1
  37. 2026Thinking Out of Order: When Output Order Stops Reflecting Reasoning Order in Diffusion Language Modelscited by 1
  38. 2026Reversible Diffusion Decoding for Diffusion Language Modelscited by 1

50 most cited of 55 All 55 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation
  2. 2020Text Generation by Learning from Demonstrations
  3. 2018Toward Diverse Text Generation with Inverse Reinforcement Learning
  4. 2025LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning
  5. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models
  6. 2020TextGAIL: Generative Adversarial Imitation Learning for Text Generation
  7. 2022Revisiting Generative Commonsense Reasoning: A Pre-Ordering Approach
  8. 2024A Critical Look At Tokenwise Reward-Guided Text Generation