# Efficient Scaling for LLM-based ASR

arXiv: https://arxiv.org/abs/2508.04096 · HTML: https://mlsanity.com/paper/2508.04096/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2025.

## Built on by (most cited first, 8 of 8)

- [Uni-ASR: Unified LLM-Based Architecture for Non-Streaming and Streaming Automatic Speech Recognition](https://mlsanity.com/paper/2603.11123/) (2026)
- [TagSpeech: End-to-End Multi-Speaker ASR and Diarization with Fine-Grained Temporal Grounding](https://mlsanity.com/paper/2601.06896/) (2026)
- [Speech LLMs are Contextual Reasoning Transcribers](https://mlsanity.com/paper/2604.00610/) (2026)
- [PAC: Pronunciation-Aware Contextualized Large Language Model-based Automatic Speech Recognition](https://mlsanity.com/paper/2509.12647/) (2025)
- [Index-MSR: A high-efficiency multimodal fusion framework for speech recognition](https://mlsanity.com/paper/2509.22744/) (2025)
- [Rethinking Speech-LLM Integration for ASR: Effective Joint Speech-Text Training by Interleaving](https://mlsanity.com/paper/2607.01733/) (2026)
- [Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs](https://mlsanity.com/paper/2609.17981/) (2026)
- [ASSET: Adaptive Speech Segmentation via Reinforcement Learning](https://mlsanity.com/paper/2610.14309/) (2026)

## Abstract (arXiv)

Large language model (LLM)-based automatic speech recognition (ASR) achieves strong performance but often incurs high computational costs. This work investigates how to obtain the best LLM-ASR performance efficiently. Through comprehensive and controlled experiments, we find that pretraining the speech encoder before integrating it with the LLM leads to significantly better scaling efficiency than the standard practice of joint post-training of LLM-ASR. Based on this insight, we propose a new multi-stage LLM-ASR training strategy, EFIN: Encoder First Integration. Among all training strategies evaluated, EFIN consistently delivers better performance (relative to 21.1% CERR) with significantly lower computation budgets (49.9% FLOPs). Furthermore, we derive a scaling law that approximates ASR error rates as a computation function, providing practical guidance for LLM-ASR scaling.