mlsanityData to 13 Oct 2026

Uni-ASR: Unified LLM-Based Architecture for Non-Streaming and Streaming Automatic Speech Recognition

arXiv 2603.11123 ↗First posted 11 Mar 2026cs.SD cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Although the deep integration of the Automatic Speech Recognition (ASR) system with Large Language Models (LLMs) has significantly improved accuracy, the deployment of such systems in low-latency streaming scenarios remains challenging. In this paper, we propose Uni-ASR, a unified framework based on LLMs that integrates both non-streaming and streaming speech recognition capabilities. We propose a joint training paradigm that enables the system to seamlessly transition between two recognition modes without any architectural modifications. Furthermore, we introduce a context-aware training paradigm and a co-designed fallback decoding strategy, which can enhance streaming recognition accuracy without introducing additional latency. The experimental results demonstrate that Uni-ASR not only achieves competitive performance within non-streaming mode, but also demonstrates strong effectiveness in streaming scenarios under diverse latency constraints.

Key facts. Cited by 5 papers in the map (data through 13 Oct 2026). Builds on 25 of its 27 references that are in the map. First posted on arXiv 11 Mar 2026.

Builds on

25 of 27 references.

References · 25

  1. 2025Qwen3 Technical Reportcited by 8,510
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,004
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,844
  4. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 459
  5. 2023Scaling Speech Technology to 1,000+ Languagescited by 402
  6. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 393
  7. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 366
  8. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 286
  9. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 191
  10. 2018AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scalecited by 125
  11. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 123
  12. 2026Qwen3-ASR Technical Reportcited by 109
13 more
  1. 2024Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognitioncited by 61
  2. 2020Low-Latency Sequence-to-Sequence Speech Recognition and Translation by Partial Hypothesis Selectioncited by 48
  3. 2022WeNet 2.0: More Productive End-to-End Speech Recognition Toolkitcited by 46
  4. 2025FireRedASR: Open-Source Industrial-Grade Mandarin Speech Recognition Models from Encoder-Decoder to LLM Integrationcited by 45
  5. 2024GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinementcited by 34
  6. 2025Streaming Sequence-to-Sequence Learning with Delayed Streams Modelingcited by 26
  7. 2025Fun-ASR Technical Reportcited by 19
  8. 2024Speech ReaLLM -- Real-time Streaming Speech Recognition with Multimodal LLMs by Teaching the Flow of Timecited by 14
  9. 2024Efficient Streaming LLM for Speech Recognitioncited by 10
  10. 2025Efficient Scaling for LLM-based ASRcited by 7
  11. 2024Transducer-Llama: Integrating LLMs into Streamable Transducer-based Speech Recognitioncited by 4
  12. 2025Index-ASR Technical Reportcited by 4
  13. 2026Streaming Speech Recognition with Decoder-Only Large Language Models and Latency Optimizationcited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026X2-Turn: Frame-Synchronous Dual-Head Modeling for Joint Streaming ASR and Turn State Predictioncited by 3
  2. 2026Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMscited by 1
  3. 2026NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASRcited by 1
  4. 2026Qwen-Audio-3.0-ASR Technical Reportcited by 1
  5. 2026X2Streaming-ASR: wait when uncertain, emit when ready for streaming ASRcited by 0