# Autoregressive Text Generation Beyond Feedback Loops

arXiv: https://arxiv.org/abs/1908.11658 · HTML: https://mlsanity.com/paper/1908.11658/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2019.

## Built on by (most cited first, 6 of 6)

- [ShuttleNet: Position-aware Fusion of Rally Progress and Player Styles for Stroke Forecasting in Badminton](https://mlsanity.com/paper/2112.01044/) (2021)
- [Recent Advances in Neural Text Generation: A Task-Agnostic Survey](https://mlsanity.com/paper/2203.03047/) (2022)
- [Prefill-level Jailbreak: A Black-Box Risk Analysis of Large Language Models](https://mlsanity.com/paper/2504.21038/) (2025)
- [Insights from Generative Modeling for Neural Video Compression](https://mlsanity.com/paper/2107.13136/) (2021)
- [Towards Unsupervised Content Disentanglement in Sentence Representations via Syntactic Roles](https://mlsanity.com/paper/2206.11184/) (2022)
- [Structured Prediction in NLP -- A survey](https://mlsanity.com/paper/2110.02057/) (2021)

## Abstract (arXiv)

Autoregressive state transitions, where predictions are conditioned on past predictions, are the predominant choice for both deterministic and stochastic sequential models. However, autoregressive feedback exposes the evolution of the hidden state trajectory to potential biases from well-known train-test discrepancies. In this paper, we combine a latent state space model with a CRF observation model. We argue that such autoregressive observation models form an interesting middle ground that expresses local correlations on the word level but keeps the state evolution non-autoregressive. On unconditional sentence generation we show performance improvements compared to RNN and GAN baselines while avoiding some prototypical failure modes of autoregressive models.