mlsanityData to 14 Oct 2026

Design of the IBM Granite 5.0 TurboCTC ASR Model

arXiv 2609.20104 ↗First posted 17 Sep 2026cs.CLcited by 0

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

We describe the architecture, training methodology and inference speedups of Granite 5.0 Turbo CTC, a 470 million parameter encoder-only model with an excellent speed-accuracy tradeoff. The architecture uses pyramidal temporal subsampling within Conformer blocks using strided depthwise convolutions, block-diagonal (chunk-wise) self-attention, and conditioning on intermediate predictions from the middle layer. Training highlights are the use of only publicly available data, the novel use of a Muon optimizer, and balanced data sampling. Inference speedups include replacing 1 x 1 convolutions with linear layers and optimizing the attention computation in the Conformer blocks. Collectively, these result in a model that is on the speed-accuracy Pareto frontier of the Open ASR leaderboard for English short-form ASR while being twice as fast as the fastest competitor. The model can be used under a permissive license and downloaded from https://huggingface.co/ibm-granite/granite-speech-5.0-470m-turboctc.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 23 of its 33 references that are in the map. First posted on arXiv 17 Sep 2026.

Builds on

23 of 33 references.

References · 23

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  4. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  5. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  6. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  7. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  8. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  9. 2025Muon is Scalable for LLM Trainingcited by 428
  10. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
  11. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 231
  12. 2024Old Optimizer, New Norm: An Anthologycited by 193
11 more
  1. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
  2. 2023Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognitioncited by 124
  3. 2026Qwen3-ASR Technical Reportcited by 118
  4. 2025The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithmcited by 110
  5. 2023SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translationcited by 76
  6. 2021Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictionscited by 61
  7. 2024YODAS: Youtube-Oriented Dataset for Audio and Speechcited by 60
  8. 2022Earnings-22: A Practical Benchmark for Accents in the Wildcited by 50
  9. 2021Efficient conformer: Progressive downsampling and grouped attention for automatic speech recognitioncited by 42
  10. 2025Granite-speech: open-source speech-aware LLMs with strong English ASR capabilitiescited by 27
  11. 2021Pushing the Limits of Non-Autoregressive Speech Recognitioncited by 18

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Deep Learning Models in Speech Recognition: Measuring GPU Energy Consumption, Impact of Noise and Model Quantization for Edge Deployment
  2. 2025Training and Inference Efficiency of Encoder-Decoder Speech Models
  3. 2022Uconv-Conformer: High Reduction of Input Sequence Length for End-to-End Speech Recognition
  4. 2025Edge-ASR: Towards Low-Bit Quantization of Automatic Speech Recognition Models
  5. 2025Unfolding A Few Structures for The Many: Memory-Efficient Compression of Conformer and Speech Foundation Models
  6. 2026Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference
  7. 2023TODM: Train Once Deploy Many Efficient Supernet-Based RNN-T Compression For On-device ASR Models
  8. 2026Test-Time Compute Scaling for ASR with Depth-Conditioned Looped Transformers