# A comparison of end-to-end models for long-form speech recognition

arXiv: https://arxiv.org/abs/1911.02242 · HTML: https://mlsanity.com/paper/1911.02242/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2019.

## Built on by (most cited first, 30 of 42)

- [Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages](https://mlsanity.com/paper/2303.01037/) (2023)
- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities](https://mlsanity.com/paper/2503.03983/) (2025)
- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [Streaming Sequence-to-Sequence Learning with Delayed Streams Modeling](https://mlsanity.com/paper/2509.08753/) (2025)
- [Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformers](https://mlsanity.com/paper/2104.09426/) (2021)
- [Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks](https://mlsanity.com/paper/2305.03101/) (2023)
- [E2E Segmenter: Joint Segmenting and Decoding for Long-Form ASR](https://mlsanity.com/paper/2204.10749/) (2022)
- [How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?](https://mlsanity.com/paper/2412.18495/) (2024)
- [Alignment Knowledge Distillation for Online Streaming Attention-based Speech Recognition](https://mlsanity.com/paper/2103.00422/) (2021)
- [Efficient Domain Adaptation for Speech Foundation Models](https://mlsanity.com/paper/2302.01496/) (2023)
- [Multilingual and Fully Non-Autoregressive ASR with Large Language Model Fusion: A Comprehensive Study](https://mlsanity.com/paper/2401.12789/) (2024)
- [Streaming End-to-End ASR based on Blockwise Non-Autoregressive Models](https://mlsanity.com/paper/2107.09428/) (2021)
- [Input Length Matters: Improving RNN-T and MWER Training for Long-form Telephony Speech Recognition](https://mlsanity.com/paper/2110.03841/) (2021)
- [Updated Corpora and Benchmarks for Long-Form Speech Recognition](https://mlsanity.com/paper/2309.15013/) (2023)
- [Streaming parallel transducer beam search with fast-slow cascaded encoders](https://mlsanity.com/paper/2203.15773/) (2022)
- [Hypothesis Stitcher for End-to-End Speaker-attributed ASR on Long-form Multi-talker Recordings](https://mlsanity.com/paper/2101.01853/) (2021)
- [Multi-mode Transformer Transducer with Stochastic Future Context](https://mlsanity.com/paper/2106.09760/) (2021)
- [Lahjoita puhetta -- a large-scale corpus of spoken Finnish with some benchmarks](https://mlsanity.com/paper/2203.12906/) (2022)
- [Chunked Attention-based Encoder-Decoder Model for Streaming Speech Recognition](https://mlsanity.com/paper/2309.08436/) (2023)
- [LongFNT: Long-form Speech Recognition with Factorized Neural Transducer](https://mlsanity.com/paper/2211.09412/) (2022)
- [Learning a Dual-Mode Speech Recognition Model via Self-Pruning](https://mlsanity.com/paper/2207.11906/) (2022)
- [Monotonic segmental attention for automatic speech recognition](https://mlsanity.com/paper/2210.14742/) (2022)
- [Survey of End-to-End Multi-Speaker Automatic Speech Recognition for Monaural Audio](https://mlsanity.com/paper/2505.10975/) (2025)
- [Memory-Efficient Training of RNN-Transducer with Sampled Softmax](https://mlsanity.com/paper/2203.16868/) (2022)
- [Random Utterance Concatenation Based Data Augmentation for Improving Short-video Speech Recognition](https://mlsanity.com/paper/2210.15876/) (2022)
- [E2E Segmentation in a Two-Pass Cascaded Encoder ASR Model](https://mlsanity.com/paper/2211.15432/) (2022)
- [A Token-Wise Beam Search Algorithm for RNN-T](https://mlsanity.com/paper/2302.14357/) (2023)
- [Long-form Simultaneous Speech Translation: Thesis Proposal](https://mlsanity.com/paper/2310.11141/) (2023)
- [How Much Context Does My Attention-Based ASR System Need?](https://mlsanity.com/paper/2310.15672/) (2023)

## Abstract (arXiv)

End-to-end automatic speech recognition (ASR) models, including both attention-based models and the recurrent neural network transducer (RNN-T), have shown superior performance compared to conventional systems. However, previous studies have focused primarily on short utterances that typically last for just a few seconds or, at most, a few tens of seconds. Whether such architectures are practical on long utterances that last from minutes to hours remains an open question. In this paper, we both investigate and improve the performance of end-to-end models on long-form transcription. We first present an empirical comparison of different end-to-end models on a real world long-form task and demonstrate that the RNN-T model is much more robust than attention-based systems in this regime. We next explore two improvements to attention-based systems that significantly improve its performance: restricting the attention to be monotonic, and applying a novel decoding algorithm that breaks long utterances into shorter overlapping segments. Combining these two improvements, we show that attention-based end-to-end models can be very competitive to RNN-T on long-form speech recognition.