mlsanityData to 7 Oct 2026

Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languages

arXiv 2310.03018 ↗First posted 4 Oct 2023eess.AS cs.CL cs.SDcited by 12

Not on the map. Why?

Abstract (arXiv)

We introduce a new zero resource code-switched speech benchmark designed to directly assess the code-switching capabilities of self-supervised speech encoders. We showcase a baseline system of language modeling on discrete units to demonstrate how the code-switching abilities of speech encoders can be assessed in a zero-resource manner. Our experiments encompass a variety of well-known speech encoders, including Wav2vec 2.0, HuBERT, XLSR, etc. We examine the impact of pre-training languages and model size on benchmark performance. Notably, though our results demonstrate that speech encoders with multilingual pre-training, exemplified by XLSR, outperform monolingual variants (Wav2vec 2.0, HuBERT) in code-switching scenarios, there is still substantial room for improvement in their code-switching linguistic abilities.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 26 of its 32 references that are in the map. First posted on arXiv 4 Oct 2023.

Builds on

26 of 32 references.

References · 26

  1. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  5. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  6. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  7. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 553
  8. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  9. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  10. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
  11. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  12. 2021Direct speech-to-speech translation with discrete unitscited by 148
14 more
  1. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
  2. 2020The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modelingcited by 100
  3. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 70
  4. 2023Multilingual Large Language Models Are Not (Yet) Code-Switcherscited by 65
  5. 2021Are Multilingual Models Effective in Code-Switching?cited by 56
  6. 2021The Zero Resource Speech Challenge 2021: Spoken language modellingcited by 47
  7. 2022Reducing Language confusion for Code-switching Speech Recognition with Token-level Language Diarizationcited by 28
  8. 2022End-to-End Speech Translation for Code Switched Speechcited by 26
  9. 2022Improving Distortion Robustness of Self-supervised Speech Processing Tasks with Domain Adaptationcited by 16
  10. 2022Code-Switching without Switching: Language Agnostic End-to-End Speech Translationcited by 13
  11. 2020Towards Natural Bilingual and Code-Switched Speech Synthesis Based on Mix of Monolingual Recordings and Cross-Lingual Voice Conversioncited by 12
  12. 2023Ensemble knowledge distillation of self-supervised speech modelscited by 10
  13. 2022Improving generalizability of distilled self-supervised speech processing models under distorted settingscited by 5
  14. 2022Benchmarking Evaluation Metrics for Code-Switching Automatic Speech Recognitioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 7; 2026: 1.

12 papers.

  1. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignmentcited by 47
  2. 2025Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Surveycited by 38
  3. 2024Speech-Copilot: Leveraging Large Language Models for Speech Processing via Task Decomposition, Modularization, and Program Generationcited by 27
  4. 2025AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Modelscited by 18
  5. 2023Investigating Zero-Shot Generalizability on Mandarin-English Code-Switched ASR and Speech-to-text Translation of Recent Foundation Models with Self-Supervision and Weak Supervisioncited by 13
  6. 2024Do Prompts Really Prompt? Exploring the Prompt Understanding Capability of Whispercited by 12
  7. 2025A Self-Refining Framework for Enhancing ASR Using TTS-Synthesized Datacited by 7
  8. 2024Leave No Knowledge Behind During Knowledge Distillation: Towards Practical and Effective Knowledge Distillation for Code-Switching ASR Using Realistic Datacited by 5
  9. 2025Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalitiescited by 4
  10. 2025SAGE: Spliced-Audio Generated Data for Enhancing Foundational Models in Low-Resource Arabic-English Code-Switched Speech Recognitioncited by 2
  11. 2026MUGEN: Evaluating and Improving Multi-audio Understanding of Large Audio-Language Modelscited by 2
  12. 2025Multilingual Information Retrieval with a Monolingual Knowledge Basecited by 1