mlsanityData to 7 Oct 2026

Insertion-based Decoding with automatically Inferred Generation Order

arXiv 1902.01370 ↗First posted 4 Feb 2019cs.CL cs.LGcited by 43

Not on the map. Why?

Abstract (arXiv)

Conventional neural autoregressive decoding commonly assumes a fixed left-to-right generation order, which may be sub-optimal. In this work, we propose a novel decoding algorithm -- InDIGO -- which supports flexible sequence generation in arbitrary orders through insertion operations. We extend Transformer, a state-of-the-art sequence generation model, to efficiently implement the proposed approach, enabling it to be trained with either a pre-defined generation order or adaptive orders obtained from beam-search. Experiments on four real-world tasks, including word order recovery, machine translation, image caption and code generation, demonstrate that our algorithm can generate sequences following arbitrary orders, while achieving competitive or even better performance compared to the conventional left-to-right generation. The generated sequences show that InDIGO adopts adaptive generation orders based on input information.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 8; 2023: 3; 2024: 6; 2025: 5; 2026: 6.

43 papers.

Sort:

By area:

  1. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  2. 2021A Survey of Transformerscited by 399
  3. 2022Efficient Training of Language Models to Fill in the Middlecited by 248
  4. 2022A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyondcited by 64
  5. 2025Edit Flows: Flow Matching with Edit Operationscited by 53
  6. 2022Why is constrained neural language generation particularly challenging?cited by 19
  7. 2024Arrows of Time for Large Language Modelscited by 18
  8. 2021Controllable Summarization with Constrained Markov Decision Processcited by 17
  9. 2021Thinking Clearly, Talking Fast: Concept-Guided Non-Autoregressive Generation for Open-Domain Dialogue Systemscited by 17
  10. 2025OneFlow: Concurrent Mixed-Modal and Interleaved Generation with Edit Flowscited by 17
  11. 2021Learning Structural Edits via Incremental Tree Transformationscited by 16
  12. 2025Any-Order GPT as Masked Diffusion Model: Decoupling Formulation and Architecturecited by 16
31 more
  1. 2026DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvascited by 16
  2. 2021Discovering Non-monotonic Autoregressive Orderings with Variational Inferencecited by 15
  3. 2021InsNet: An Efficient, Flexible, and Performant Insertion-based Text Generation Modelcited by 14
  4. 2024The Belief State Transformercited by 12
  5. 2021Exploring Dynamic Selection of Branch Expansion Orders for Code Generationcited by 10
  6. 2021Leveraging Recursive Gumbel-Max Trick for Approximate Inference in Combinatorial Spacescited by 7
  7. 2022UCEpic: Unifying Aspect Planning and Lexical Constraints for Generating Explanations in Recommendationcited by 6
  8. 2021Data-to-text Generation by Splicing Together Nearest Neighborscited by 4
  9. 2022Align, Write, Re-order: Explainable End-to-End Speech Translation via Operation Sequence Generationcited by 4
  10. 2021Extending Multi-Sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applicationscited by 3
  11. 2021Ultra-high Resolution Image Segmentation via Locality-aware Context Fusion and Alternating Local Enhancementcited by 3
  12. 2023Self-Infilling Code Generationcited by 3
  13. 2024Returning to the Start: Generating Narratives with Related Endpointscited by 3
  14. 2024BLens: Contrastive Captioning of Binary Functions using Ensemble Embeddingcited by 3
  15. 2021Diformer: Directional Transformer for Neural Machine Translationcited by 2
  16. 2022Modeling Spatial Trajectories using Coarse-Grained Smartphone Logscited by 2
  17. 2022Easy Guided Decoding in Providing Suggestions for Interactive Machine Translationcited by 2
  18. 2023Interactive Editing for Text Summarizationcited by 2
  19. 2024Improving Discrete Diffusion Models via Structured Preferential Generationcited by 2
  20. 2024Reverse Modeling in Large Language Modelscited by 2
  21. 2026Variational Learning for Insertion-based Generationcited by 2
  22. 2021Towards More Efficient Insertion Transformer with Fractional Positional Encodingcited by 1
  23. 2023Deep Generative Models for Decision-Making and Controlcited by 1
  24. 2025What Makes the Preferred Thinking Direction for LLMs in Multiple-choice Questions?cited by 1
  25. 2025Distilling Specialized Orders for Visual Generationcited by 1
  26. 2026Thinking Out of Order: When Output Order Stops Reflecting Reasoning Order in Diffusion Language Modelscited by 1
  27. 2026Insertion Based Sequence Generation with Learnable Order Dynamicscited by 1
  28. 2026Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Modelscited by 1
  29. 2021Learning and Analyzing Generation Order for Undirected Sequence Modelscited by 0
  30. 2022P3LM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-Trainingcited by 0
  31. 2026DeMTS: Denoising Trajectories as Multivariate Time Series for Hallucination Detection in Diffusion Language Modelscited by 0