# Equivalence of Segmental and Neural Transducer Modeling: A Proof of Concept

arXiv: https://arxiv.org/abs/2104.06104 · HTML: https://mlsanity.com/paper/2104.06104/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 14 of 14)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Hybrid Autoregressive Transducer (hat)](https://mlsanity.com/paper/2003.07705/) (2020)
- [Improved training of end-to-end attention models for speech recognition](https://mlsanity.com/paper/1805.03294/) (2018)
- [Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboard](https://mlsanity.com/paper/2001.07263/) (2020)
- [A New Training Pipeline for an Improved Neural Transducer](https://mlsanity.com/paper/2005.09319/) (2020)
- [Phoneme Based Neural Transducer for Large Vocabulary Speech Recognition](https://mlsanity.com/paper/2010.16368/) (2020)
- [RETURNN: The RWTH Extensible Training framework for Universal Recurrent Neural Networks](https://mlsanity.com/paper/1608.00895/) (2016)
- [Accelerating RNN Transducer Inference via One-Step Constrained Beam Search](https://mlsanity.com/paper/2002.03577/) (2020)
- [A Comprehensive Study of Deep Bidirectional LSTM RNNs for Acoustic Modeling in Speech Recognition](https://mlsanity.com/paper/1606.06871/) (2016)
- [A study of latent monotonic attention variants](https://mlsanity.com/paper/2103.16710/) (2021)
- [Context-Dependent Acoustic Modeling without Explicit Phone Clustering](https://mlsanity.com/paper/2005.07578/) (2020)

## Built on by (most cited first, 5 of 5)

- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [Chunked Attention-based Encoder-Decoder Model for Streaming Speech Recognition](https://mlsanity.com/paper/2309.08436/) (2023)
- [Monotonic segmental attention for automatic speech recognition](https://mlsanity.com/paper/2210.14742/) (2022)
- [RASR2: The RWTH ASR Toolkit for Generic Sequence-to-sequence Speech Recognition](https://mlsanity.com/paper/2305.17782/) (2023)
- [Token-Weighted RNN-T for Learning from Flawed Data](https://mlsanity.com/paper/2406.18108/) (2024)

## Abstract (arXiv)

With the advent of direct models in automatic speech recognition (ASR), the formerly prevalent frame-wise acoustic modeling based on hidden Markov models (HMM) diversified into a number of modeling architectures like encoder-decoder attention models, transducer models and segmental models (direct HMM). While transducer models stay with a frame-level model definition, segmental models are defined on the level of label segments directly. While (soft-)attention-based models avoid explicit alignment, transducer and segmental approach internally do model alignment, either by segment hypotheses or, more implicitly, by emitting so-called blank symbols. In this work, we prove that the widely used class of RNN-Transducer models and segmental models (direct HMM) are equivalent and therefore show equal modeling power. It is shown that blank probabilities translate into segment length probabilities and vice versa. In addition, we provide initial experiments investigating decoding and beam-pruning, comparing time-synchronous and label-/segment-synchronous search strategies and their properties using the same underlying model.