# Sequence Modeling with Unconstrained Generation Order

arXiv: https://arxiv.org/abs/1911.00176 · HTML: https://mlsanity.com/paper/1911.00176/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2019.

## Built on by (most cited first, 8 of 8)

- [Generative Flow Networks for Discrete Probabilistic Modeling](https://mlsanity.com/paper/2202.01361/) (2022)
- [Training and Inference on Any-Order Autoregressive Models the Right Way](https://mlsanity.com/paper/2205.13554/) (2022)
- [Arrows of Time for Large Language Models](https://mlsanity.com/paper/2401.17505/) (2024)
- [PAM: Understanding Product Images in Cross Product Category Attribute Extraction](https://mlsanity.com/paper/2106.04630/) (2021)
- [Discovering Non-monotonic Autoregressive Orderings with Variational Inference](https://mlsanity.com/paper/2110.15797/) (2021)
- [InfoDiffusion: Information Entropy Aware Diffusion Process for Non-Autoregressive Text Generation](https://mlsanity.com/paper/2310.11976/) (2023)
- [Align, Write, Re-order: Explainable End-to-End Speech Translation via Operation Sequence Generation](https://mlsanity.com/paper/2211.05967/) (2022)
- [P3LM: Probabilistically Permuted Prophet Language Modeling for Generative Pre-Training](https://mlsanity.com/paper/2210.12339/) (2022)

## Abstract (arXiv)

The dominant approach to sequence generation is to produce a sequence in some predefined order, e.g. left to right. In contrast, we propose a more general model that can generate the output sequence by inserting tokens in any arbitrary order. Our model learns decoding order as a result of its training procedure. Our experiments show that this model is superior to fixed order models on a number of sequence generation tasks, such as Machine Translation, Image-to-LaTeX and Image Captioning.