# Speech Encoder Fusion for LLM-based Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2606.10853 · HTML: https://mlsanity.com/paper/2606.10853/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2026.

## Built on by (most cited first, 1 of 1)

- [Correlation-Guided Encoder Selection for Multi-Encoder Large Audio-Language Models](https://mlsanity.com/paper/2609.18041/) (2026)

## Abstract (arXiv)

Speech-aware large language models (LLMs) can incorporate speech through pre-trained acoustic encoders that project speech features into the LLM embedding space. While the choice of the speech encoder critically influences performance, different encoders often exhibit complementary strengths, motivating their combination. In this work, we investigate whether fusing multiple pre-trained speech encoders can enhance speech-aware LLMs for automatic speech recognition (ASR). We explore several fusion strategies beyond simple feature concatenation, including learned combinations and Transformer-based fusion architectures, and evaluate them across mono- and multilingual ASR settings as well as diarized speech recognition. Our results indicate that carefully fusing multiple parallel speech encoders improves downstream performance in all scenarios with limited computational overhead.