mlsanityData to 14 Oct 2026

Dolphin: A Large-Scale Automatic Speech Recognition Model for Eastern Languages

arXiv 2503.20212 ↗First posted 26 Mar 2025cs.CL eess.AScited by 8

Not on the map. Why?

Abstract (arXiv)

This report introduces Dolphin, a large-scale multilingual automatic speech recognition (ASR) model that extends the Whisper architecture to support a wider range of languages. Our approach integrates in-house proprietary and open-source datasets to refine and optimize Dolphin's performance. The model is specifically designed to achieve notable recognition accuracy for 40 Eastern languages across East Asia, South Asia, Southeast Asia, and the Middle East, while also supporting 22 Chinese dialects. Experimental evaluations show that Dolphin significantly outperforms current state-of-the-art open-source models across various languages. To promote reproducibility and community-driven innovation, we are making our trained models and inference source code publicly available.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 38 of its 51 references that are in the map. First posted on arXiv 26 Mar 2025.

Builds on

38 of 51 references.

References · 38

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Layer Normalizationcited by 5,669
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  7. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  8. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  9. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  10. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  11. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  12. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
26 more
  1. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  2. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  3. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  4. 2015Listen, Attend and Spellcited by 543
  5. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  6. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  7. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  8. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  9. 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 451
  10. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  11. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
  12. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 257
  13. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 231
  14. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 226
  15. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 210
  16. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  17. 2022Self-supervised Learning with Random-projection Quantizer for Speech Recognitioncited by 162
  18. 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
  19. 2014Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognitioncited by 111
  20. 2022Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Languagecited by 110
  21. 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 92
  22. 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 74
  23. 2024YODAS: Youtube-Oriented Dataset for Audio and Speechcited by 60
  24. 2023Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Datacited by 42
  25. 2024GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinementcited by 34
  26. 2021Wav2vec-C: A Self-supervised Model for Speech Representation Learningcited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 3.

8 papers.

  1. 2025Fun-ASR Technical Reportcited by 19
  2. 2025ContextASR-Bench: A Massive Contextual Speech Recognition Benchmarkcited by 15
  3. 2025Adaptive Shared Experts with LoRA-Based Mixture of Experts for Multi-Task Learningcited by 3
  4. 2025ILT-Iterative LoRA Training through Focus-Feedback-Fix for Multilingual Speech Recognitioncited by 1
  5. 2025Listening, Imagining & Refining: A Heuristic Optimized ASR Correction Framework with LLMscited by 1
  6. 2026Dolphin-CN-Dialect: Where Chinese Dialects Mattercited by 1
  7. 2026Bridging the Stability-Expressivity Gap: Synthetic Data Scaling and Preference Alignment for Low-Resource Spoken Language Modelscited by 0
  8. 2026Evaluating Bias in Phoneme-Based Automatic Speech Recognition Systems: An Analysis of IPA Transcription Modelscited by 0