mlsanityData to 7 Oct 2026

BiMamba2 Masked Discrete-Unit Prediction for Multilingual Speech Representation for Unsupervised Speech in the Wild Challenge

arXiv 2609.28758 ↗First posted 23 Sep 2026cs.SD cs.CLcited by 0

On the map: State-space sequence models (S4, Mamba) → Speech and audio SSMs

Abstract (arXiv)

We describe our submission to the Unsupervised Speech in the Wild (UPS) Challenge at Interspeech 2026, a bidirectional Mamba-2 (BiMamba2) encoder trained with masked discrete-unit prediction following the HuBERT-style paradigm. The 47.88M-parameter model is trained on 250 hours of speech across 67 languages from the MLCommons Unsupervised People's Speech dataset, with no labeled data. The objective combines masked k-means pseudo-label prediction with language identification supervision and VICReg regularization. On official evaluation, the system achieves an Adjusted Rand Index of 0.735, exceeding four baselines on speaker clustering. Language identification macro-F1 (0.073) and character error rate (0.870) remain below supervised baselines. We analyze a local-official discrepancy in metric scale and checkpoint ranking, highlighting limitations of in-distribution diagnostics for predicting Dynabench probe outcomes.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 20 of its 25 references that are in the map. First posted on arXiv 23 Sep 2026.

Builds on

20 of 25 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  7. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  8. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  9. 2019Root Mean Square Layer Normalizationcited by 1,425
  10. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  11. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  12. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
8 more
  1. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  2. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  3. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  4. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
  5. 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 378
  6. 2021The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usagecited by 98
  7. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 70
  8. 2024Hydra: Bidirectional State Space Models Through Generalized Matrix Mixerscited by 43

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Monolingual to Multilingual: Evaluating Mamba for ASR in South African Languages
  2. 2025Speech Language Models for Under-Represented Languages: Insights from Wolof
  3. 2025MLMA: Towards Multilingual ASR With Mamba-based Architectures
  4. 2024BEST-STD: Bidirectional Mamba-Enhanced Speech Tokenization for Spoken Term Detection
  5. 2026Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
  6. 2026TC-BiMamba: Trans-Chunk bidirectionally within BiMamba for unified streaming and non-streaming ASR
  7. 2025AfriSpeech-MultiBench: A Verticalized Multidomain Multicountry Benchmark Suite for African Accented English ASR
  8. 2026PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialects