# A Transcription Prompt-based Efficient Audio Large Language Model for Robust Speech Recognition

arXiv: https://arxiv.org/abs/2408.09491 · HTML: https://mlsanity.com/paper/2408.09491/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Aug 2024.

## Built on by (most cited first, 3 of 3)

- [Enhancing Dialogue Annotation with Speaker Characteristics Leveraging a Frozen LLM](https://mlsanity.com/paper/2508.04795/) (2025)
- [MFLA: Monotonic Finite Look-ahead Attention for Streaming Speech Recognition](https://mlsanity.com/paper/2506.03722/) (2025)
- [Investigating the Viability of Employing Multi-modal Large Language Models in the Context of Audio Deepfake Detection](https://mlsanity.com/paper/2601.00777/) (2026)

## Abstract (arXiv)

Audio-LLM introduces audio modality into a large language model (LLM) to enable a powerful LLM to recognize, understand, and generate audio. However, during speech recognition in noisy environments, we observed the presence of illusions and repetition issues in audio-LLM, leading to substitution and insertion errors. This paper proposes a transcription prompt-based audio-LLM by introducing an ASR expert as a transcription tokenizer and a hybrid Autoregressive (AR) Non-autoregressive (NAR) decoding approach to solve the above problems. Experiments on 10k-hour WenetSpeech Mandarin corpus show that our approach decreases 12.2% and 9.6% CER relatively on Test_Net and Test_Meeting evaluation sets compared with baseline. Notably, we reduce the decoding repetition rate on the evaluation set to zero, showing that the decoding repetition problem has been solved fundamentally.