mlsanityData to 7 Oct 2026

MERaLiON-SpeechEncoder: Towards a Speech Foundation Model for Singapore and Beyond

arXiv 2412.11538 ↗First posted 16 Dec 2024cs.CL cs.AI eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

This technical report describes the MERaLiON-SpeechEncoder, a foundation model designed to support a wide range of downstream speech applications. Developed as part of Singapore's National Multimodal Large Language Model Programme, the MERaLiON-SpeechEncoder is tailored to address the speech processing needs in Singapore and the surrounding Southeast Asian region. The model currently supports mainly English, including the variety spoken in Singapore. We are actively expanding our datasets to gradually cover other languages in subsequent releases. The MERaLiON-SpeechEncoder was pre-trained from scratch on 200,000 hours of unlabelled speech data using a self-supervised learning approach based on masked language modelling. We describe our training procedure and hyperparameter tuning experiments in detail below. Our evaluation demonstrates improvements to spontaneous and Singapore speech benchmarks for speech recognition, while remaining competitive to other state-of-the-art speech encoders across ten other speech tasks. We commit to releasing our model, supporting broader research endeavours, both in Singapore and beyond.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 28 of its 35 references that are in the map. First posted on arXiv 16 Dec 2024.

Builds on

28 of 35 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  7. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  9. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  10. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  11. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  12. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
16 more
  1. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  2. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  3. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  4. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  5. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  6. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  7. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  8. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
  9. 2022Self-supervised Learning with Random-projection Quantizer for Speech Recognitioncited by 158
  10. 2021The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usagecited by 98
  11. 2021TorchAudio: Building Blocks for Audio and Speech Processingcited by 83
  12. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 76
  13. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 70
  14. 2024Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognitioncited by 61
  15. 2021Self-Supervised Learning for speech recognition with Intermediate layer supervisioncited by 15
  16. 2024Anatomy of Industrial Scale Multilingual ASRcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofingcited by 17
  2. 2025Contextual Paralinguistic Data Creation for Multi-Modal Speech-LLM: Data Condensation and Spoken QA Generationcited by 4
  3. 2025MultiGen: Child-Friendly Multilingual Speech Generator with LLMscited by 1
  4. 2026Alethia: A Foundational Encoder for Voice Deepfakescited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024MERaLiON-AudioLLM: Bridging Audio and Language with Large Language Models
  2. 2026SEA-SpeechBench: A Large-Scale Multitask Benchmark for Speech Understanding Across Southeast Asia
  3. 2025MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages
  4. 2026Efficiently Adapting Spoken Language Models for the Singaporean Context
  5. 2024MERaLiON-TextLLM: Cross-Lingual Understanding of Large Language Models in Chinese, Indonesian, Malay, and Singlish
  6. 2025Triple X: A LLM-Based Multilingual Speech Recognition System for the INTERSPEECH2025 MLC-SLM Challenge
  7. 2026NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR
  8. 2025Speech Language Models for Under-Represented Languages: Insights from Wolof