# Cascaded Text Generation with Markov Transformers

arXiv: https://arxiv.org/abs/2006.01112 · HTML: https://mlsanity.com/paper/2006.01112/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2020.

## Built on by (most cited first, 7 of 7)

- [A Character-Level Length-Control Algorithm for Non-Autoregressive Sentence Summarization](https://mlsanity.com/paper/2205.14522/) (2022)
- [Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering](https://mlsanity.com/paper/2410.01660/) (2024)
- [Learning with Instance Bundles for Reading Comprehension](https://mlsanity.com/paper/2104.08735/) (2021)
- [Revisiting the Markov Property for Machine Translation](https://mlsanity.com/paper/2402.02084/) (2024)
- [Pipelined Decoder for Efficient Context-Aware Text Generation](https://mlsanity.com/paper/2506.23431/) (2025)
- [Structured Prediction in NLP -- A survey](https://mlsanity.com/paper/2110.02057/) (2021)
- [Fast Inference via Hierarchical Speculative Decoding](https://mlsanity.com/paper/2510.19705/) (2025)

## Abstract (arXiv)

The two dominant approaches to neural text generation are fully autoregressive models, using serial beam search decoding, and non-autoregressive models, using parallel decoding with no output dependencies. This work proposes an autoregressive model with sub-linear parallel time generation. Noting that conditional random fields with bounded context can be decoded in parallel, we propose an efficient cascaded decoding approach for generating high-quality output. To parameterize this cascade, we introduce a Markov transformer, a variant of the popular fully autoregressive model that allows us to simultaneously decode with specific autoregressive context cutoffs. This approach requires only a small modification from standard autoregressive training, while showing competitive accuracy/speed tradeoff compared to existing methods on five machine translation datasets.