# A Generalized Formalism of Auto-Regressive Decoding for Speech Processing

arXiv: https://arxiv.org/abs/2606.20714 · HTML: https://mlsanity.com/paper/2606.20714/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 16 Jun 2026.

## Builds on (most cited first, 30 of 32)

- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Faith and Fate: Limits of Transformers on Compositionality](https://mlsanity.com/paper/2305.18654/) (2023)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [Blockwise Parallel Decoding for Deep Autoregressive Models](https://mlsanity.com/paper/1811.03115/) (2018)
- [Beam Search Strategies for Neural Machine Translation](https://mlsanity.com/paper/1702.01806/) (2017)
- [The pitfalls of next-token prediction](https://mlsanity.com/paper/2403.06963/) (2024)
- [Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translation](https://mlsanity.com/paper/2005.10283/) (2020)
- [From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models](https://mlsanity.com/paper/2406.16838/) (2024)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation](https://mlsanity.com/paper/2005.03642/) (2020)
- [Accelerating Transformer Inference for Translation via Parallel Decoding](https://mlsanity.com/paper/2305.10427/) (2023)
- [Truncation Sampling as Language Model Desmoothing](https://mlsanity.com/paper/2210.15191/) (2022)
- [VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild](https://mlsanity.com/paper/2403.16973/) (2024)
- [Learning to Write with Cooperative Discriminators](https://mlsanity.com/paper/1805.06087/) (2018)
- [A Thorough Examination of Decoding Methods in the Era of LLMs](https://mlsanity.com/paper/2402.06925/) (2024)
- [Locally Typical Sampling](https://mlsanity.com/paper/2202.00666/) (2022)
- [Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation](https://mlsanity.com/paper/2204.01171/) (2022)
- [On Decoding Strategies for Neural Text Generators](https://mlsanity.com/paper/2203.15721/) (2022)
- [Comparison of Diverse Decoding Methods from Conditional Language Models](https://mlsanity.com/paper/1906.06362/) (2019)
- [A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond](https://mlsanity.com/paper/2204.09269/) (2022)
- [Machine Translation Decoding beyond Beam Search](https://mlsanity.com/paper/2104.05336/) (2021)
- [Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding](https://mlsanity.com/paper/2106.04970/) (2021)
- [Grammatical Error Correction: A Survey of the State of the Art](https://mlsanity.com/paper/2211.05166/) (2022)
- [Best-First Beam Search](https://mlsanity.com/paper/2007.03909/) (2020)
- [Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Survey](https://mlsanity.com/paper/2412.06602/) (2024)
- [Decoding Methods for Neural Narrative Generation](https://mlsanity.com/paper/2010.07375/) (2020)
- [Pseudo-Autoregressive Neural Codec Language Models for Efficient Zero-Shot Text-to-Speech Synthesis](https://mlsanity.com/paper/2504.10352/) (2025)
- [CTC-based Non-autoregressive Speech Translation](https://mlsanity.com/paper/2305.17358/) (2023)
- [End-to-End Speech-to-Text Translation: A Survey](https://mlsanity.com/paper/2312.01053/) (2023)

## Abstract (arXiv)

In speech processing, most state-of-the-art sequence prediction models rely on auto-regressive (AR) strategies to generate output sequences based on the raw predictions of the model. Despite their crucial role in the inference process, a comprehensive overview of AR strategies as a unified field is lacking, due largely to implicit and multiple definitions of next-token decoding. This context complicates the choice, comparison, and evaluation of strategies, while creating inconsistencies in the characterization of approaches as auto-regressive or not. We begin by setting explicit inclusion criteria for the field of AR search in speech processing, and derive a generalized theoretical framework to categorize and report on search strategies for neural models. We show the capabilities of this formalism in simplifying the design of benchmarks centered around the decoding process, allowing for ablation studies that are focused on search strategies.