# Investigating End-to-End ASR Architectures for Long Form Audio Transcription

arXiv: https://arxiv.org/abs/2309.09950 · HTML: https://mlsanity.com/paper/2309.09950/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2023.

## Built on by (most cited first, 11 of 11)

- [OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identification](https://mlsanity.com/paper/2402.12654/) (2024)
- [Open ASR Leaderboard: Towards Reproducible and Transparent Multilingual and Long-Form Speech Recognition Evaluation](https://mlsanity.com/paper/2510.06961/) (2025)
- [SONICS: Synthetic Or Not -- Identifying Counterfeit Songs](https://mlsanity.com/paper/2408.14080/) (2024)
- [Longer is (Not Necessarily) Stronger: Punctuated Long-Sequence Training for Enhanced Speech Recognition and Translation](https://mlsanity.com/paper/2409.05601/) (2024)
- [How Much Context Does My Attention-Based ASR System Need?](https://mlsanity.com/paper/2310.15672/) (2023)
- [Joint Fine-tuning and Conversion of Pretrained Speech and Language Models towards Linear Complexity](https://mlsanity.com/paper/2410.06846/) (2024)
- [Re-evaluating Minimum Bayes Risk Decoding for Automatic Speech Recognition](https://mlsanity.com/paper/2510.19471/) (2025)
- [Less is More: Accurate Speech Recognition & Translation without Web-Scale Data](https://mlsanity.com/paper/2406.19674/) (2024)
- [Accurate, fast, cheap: Choose three. Replacing Multi-Head-Attention with Bidirectional Recurrent Attention for Long-Form ASR](https://mlsanity.com/paper/2506.19761/) (2025)
- [TRADE: Transducer-Augmented Decoder for Speech LLM](https://mlsanity.com/paper/2606.08486/) (2026)
- [FBK's Long-form SpeechLLMs for IWSLT 2026 Instruction Following](https://mlsanity.com/paper/2606.26819/) (2026)

## Abstract (arXiv)

This paper presents an overview and evaluation of some of the end-to-end ASR models on long-form audios. We study three categories of Automatic Speech Recognition(ASR) models based on their core architecture: (1) convolutional, (2) convolutional with squeeze-and-excitation and (3) convolutional models with attention. We selected one ASR model from each category and evaluated Word Error Rate, maximum audio length and real-time factor for each model on a variety of long audio benchmarks: Earnings-21 and 22, CORAAL, and TED-LIUM3. The model from the category of self-attention with local attention and global token has the best accuracy comparing to other architectures. We also compared models with CTC and RNNT decoders and showed that CTC-based models are more robust and efficient than RNNT on long form audio.