# Global Normalization for Streaming Speech Recognition in a Modular Framework

arXiv: https://arxiv.org/abs/2205.13674 · HTML: https://mlsanity.com/paper/2205.13674/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 46 references that are in the map.
- First posted on arXiv 26 May 2022.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss](https://mlsanity.com/paper/2002.02562/) (2020)
- [Hybrid Autoregressive Transducer (hat)](https://mlsanity.com/paper/2003.07705/) (2020)
- [Wav2Letter: an End-to-End ConvNet-based Speech Recognition System](https://mlsanity.com/paper/1609.03193/) (2016)
- [Globally Normalized Transition-Based Neural Networks](https://mlsanity.com/paper/1603.06042/) (2016)
- [An Empirical Investigation of Global and Local Normalization for Recurrent Neural Sequence Models Using a Continuous Relaxation to Beam Search](https://mlsanity.com/paper/1904.06834/) (2019)
- [End-to-End Neural Segmental Models for Speech Recognition](https://mlsanity.com/paper/1708.00531/) (2017)
- [Advancing CTC-CRF Based End-to-End Speech Recognition with Wordpieces and Conformers](https://mlsanity.com/paper/2107.03007/) (2021)
- [Discriminative Segmental Cascades for Feature-Rich Phone Recognition](https://mlsanity.com/paper/1507.06073/) (2015)

## Built on by (most cited first, 10 of 10)

- [Powerful and Extensible WFST Framework for RNN-Transducer Losses](https://mlsanity.com/paper/2303.10384/) (2023)
- [Preference Optimization for Molecule Synthesis with Conditional Residual Energy-based Models](https://mlsanity.com/paper/2406.02066/) (2024)
- [Exploring Energy-based Language Models with Different Architectures and Training Methods for Speech Recognition](https://mlsanity.com/paper/2305.12676/) (2023)
- [Alignment Entropy Regularization](https://mlsanity.com/paper/2212.12442/) (2022)
- [LAST: Scalable Lattice-Based Speech Modelling in JAX](https://mlsanity.com/paper/2304.13134/) (2023)
- [Globally Normalising the Transducer for Streaming Speech Recognition](https://mlsanity.com/paper/2307.10975/) (2023)
- [Towards Maximum Likelihood Training for Transducer-based Streaming Speech Recognition](https://mlsanity.com/paper/2411.17537/) (2024)
- [Do we really need Self-Attention for Streaming Automatic Speech Recognition?](https://mlsanity.com/paper/2601.19960/) (2026)
- [SENS-ASR: Semantic Embedding injection in Neural-transducer for Streaming Automatic Speech Recognition](https://mlsanity.com/paper/2603.10005/) (2026)
- [Streaming Speech-to-Text Translation with a SpeechLLM](https://mlsanity.com/paper/2605.14766/) (2026)

## Abstract (arXiv)

We introduce the Globally Normalized Autoregressive Transducer (GNAT) for addressing the label bias problem in streaming speech recognition. Our solution admits a tractable exact computation of the denominator for the sequence-level normalization. Through theoretical and empirical results, we demonstrate that by switching to a globally normalized model, the word error rate gap between streaming and non-streaming speech-recognition models can be greatly reduced (by more than 50% on the Librispeech dataset). This model is developed in a modular framework which encompasses all the common neural speech recognition models. The modularity of this framework enables controlled comparison of modelling choices and creation of new models.