# Bi-directional Context-Enhanced Speech Large Language Models for Multilingual Conversational ASR

arXiv: https://arxiv.org/abs/2506.13396 · HTML: https://mlsanity.com/paper/2506.13396/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 19 of 19)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [WavLLM: Towards Robust and Adaptive Speech Large Language Model](https://mlsanity.com/paper/2404.00656/) (2024)
- [An Embarrassingly Simple Approach for LLM with Strong ASR Capacity](https://mlsanity.com/paper/2402.08846/) (2024)
- [Step-Audio: Unified Understanding and Generation in Intelligent Speech Interaction](https://mlsanity.com/paper/2502.11946/) (2025)
- [GenSE: Generative Speech Enhancement via Language Models using Hierarchical Modeling](https://mlsanity.com/paper/2502.02942/) (2025)
- [PromptASR for contextualized ASR with controllable style](https://mlsanity.com/paper/2309.07414/) (2023)
- [MERLIon CCS Challenge: A English-Mandarin code-switching child-directed speech corpus for language identification and diarization](https://mlsanity.com/paper/2305.18881/) (2023)
- [Conversational Speech Recognition By Learning Conversation-level Characteristics](https://mlsanity.com/paper/2202.07855/) (2022)
- [MERaLiON-AudioLLM: Bridging Audio and Language with Large Language Models](https://mlsanity.com/paper/2412.09818/) (2024)
- [SpeechT-RAG: Reliable Depression Detection in LLMs with Retrieval-Augmented Generation Using Speech Timing Information](https://mlsanity.com/paper/2502.10950/) (2025)
- [BERT Attends the Conversation: Improving Low-Resource Conversational ASR](https://mlsanity.com/paper/2110.02267/) (2021)
- [Conversational Speech Recognition by Learning Audio-textual Cross-modal Contextual Representation](https://mlsanity.com/paper/2310.14278/) (2023)

## Abstract (arXiv)

This paper introduces the integration of language-specific bi-directional context into a speech large language model (SLLM) to improve multilingual continuous conversational automatic speech recognition (ASR). We propose a character-level contextual masking strategy during training, which randomly removes portions of the context to enhance robustness and better emulate the flawed transcriptions that may occur during inference. For decoding, a two-stage pipeline is utilized: initial isolated segment decoding followed by context-aware re-decoding using neighboring hypotheses. Evaluated on the 1500-hour Multilingual Conversational Speech and Language Model (MLC-SLM) corpus covering eleven languages, our method achieves an 18% relative improvement compared to a strong baseline, outperforming even the model trained on 6000 hours of data for the MLC-SLM competition. These results underscore the significant benefit of incorporating contextual information in multilingual continuous conversational ASR.