mlsanityData to 14 Oct 2026

SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning

arXiv 2210.08634 ↗First posted 16 Oct 2022cs.CL cs.SD eess.AScited by 26

Not on the map. Why?

Abstract (arXiv)

We present the SUPERB challenge at SLT 2022, which aims at learning self-supervised speech representation for better performance, generalization, and efficiency. The challenge builds upon the SUPERB benchmark and implements metrics to measure the computation requirements of self-supervised learning (SSL) representation and to evaluate its generalizability and performance across the diverse SUPERB tasks. The SUPERB benchmark provides comprehensive coverage of popular speech processing tasks, from speech and speaker recognition to audio generation and semantic understanding. As SSL has gained interest in the speech community and showed promising outcomes, we envision the challenge to uplevel the impact of SSL techniques by motivating more practical designs of techniques beyond task performance. We summarize the results of 14 submitted models in this paper. We also discuss the main findings from those submissions and the future directions of SSL research.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 56 of its 61 references that are in the map. First posted on arXiv 16 Oct 2022.

Builds on

56 of 61 references.

References · 56

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  9. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  10. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  11. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  12. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
38 more
  1. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  2. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 425
  3. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 340
  4. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
  5. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 198
  6. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 188
  7. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 187
  8. 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 186
  9. 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 151
  10. 2021Unsupervised Speech Recognitioncited by 148
  11. 2021DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERTcited by 144
  12. 2020Multi-task self-supervised learning for Robust Speech Recognitioncited by 130
  13. 2019Learning Problem-agnostic Speech Representations from Multiple Self-supervised Taskscited by 87
  14. 2019Generative Pre-Training for Speech with Autoregressive Predictive Codingcited by 82
  15. 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
  16. 2019CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognitioncited by 77
  17. 2020DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector Quantizationcited by 77
  18. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 77
  19. 2020Vector-Quantized Autoregressive Predictive Codingcited by 69
  20. 2019Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognitioncited by 66
  21. 2020Audio ALBERT: A Lite BERT for Self-supervised Learning of Audio Representationcited by 59
  22. 2020Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challengecited by 55
  23. 2022LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERTcited by 50
  24. 2020Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependenciescited by 47
  25. 2021LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speechcited by 47
  26. 2021UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Trainingcited by 45
  27. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 45
  28. 2020The Zero Resource Speech Challenge 2020: Discovering discrete subword and word unitscited by 39
  29. 2019Improving Transformer-based Speech Recognition Using Unsupervised Pre-trainingcited by 30
  30. 2022SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Modelcited by 29
  31. 2022Exploring Efficient-tuning Methods in Self-supervised Speech Modelscited by 26
  32. 2022Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modelingcited by 25
  33. 2020Improved Speech Representations with Multi-Target Autoregressive Predictive Codingcited by 23
  34. 2020Transformer VQ-VAE for Unsupervised Unit Discovery and Speech Synthesis: ZeroSpeech 2020 Challengecited by 21
  35. 2019Speech-XLNet: Unsupervised Acoustic Model Pretraining For Self-Attention Networkscited by 17
  36. 2022Improving Distortion Robustness of Self-supervised Speech Processing Tasks with Domain Adaptationcited by 17
  37. 2020A Further Study of Unsupervised Pre-training for Transformer Based Speech Recognitioncited by 12
  38. 2021Analyzing the Robustness of Unsupervised Speech Recognitioncited by 11

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 14; 2024: 4; 2025: 5; 2026: 1.

26 papers.

Sort:

By area:

  1. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 71
  2. 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 40
  3. 2023Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?cited by 28
  4. 2023CMOT: Cross-modal Mixup via Optimal Transport for Speech Translationcited by 22
  5. 2023SpeechGen: Unlocking the Generative Power of Speech Language Models with Promptscited by 21
  6. 2023AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Modelscited by 19
  7. 2023Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Headscited by 17
  8. 2022MelHuBERT: A simplified HuBERT on Mel spectrogramscited by 16
  9. 2023Fine-tuning Strategies for Faster Inference using Speech Self-Supervised Models: A Comparative Studycited by 15
  10. 2025Leveraging Allophony in Self-Supervised Speech Models for Atypical Pronunciation Assessmentcited by 15
  11. 2023Ensemble knowledge distillation of self-supervised speech modelscited by 12
  12. 2023Findings of the 2023 ML-SUPERB Challenge: Pre-Training and Evaluation over More Languages and Beyondcited by 10
14 more
  1. 2023CoLLD: Contrastive Layer-to-layer Distillation for Compressing Multilingual Pre-trained Speech Encoderscited by 9
  2. 2023MiniSUPERB: Lightweight Benchmark for Self-supervised Speech Modelscited by 8
  3. 2024Wav2Gloss: Generating Interlinear Glossed Text from Speechcited by 8
  4. 2025USAD: Universal Speech and Audio Representation via Distillationcited by 7
  5. 2023Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillationcited by 6
  6. 2024Property Neurons in Self-Supervised Speech Transformerscited by 5
  7. 2024DAISY: Data Adaptive Self-Supervised Early Exit for Speech Representation Modelscited by 4
  8. 2025The ML-SUPERB 2.0 Challenge: Towards Inclusive ASR Benchmarking for All Language Varietiescited by 4
  9. 2022Introducing Semantics into Speech Encoderscited by 3
  10. 2023STaR: Distilling Speech Temporal Relation for Lightweight Speech Self-Supervised Learning Modelscited by 3
  11. 2025Speech-FT: Merging Pre-trained And Fine-Tuned Speech Representation Models For Cross-Task Generalizationcited by 2
  12. 2025An Exploration of Mamba for Speech Self-Supervised Modelscited by 2
  13. 2023QS-TTS: Towards Semi-Supervised Text-to-Speech Synthesis via Vector-Quantized Self-Supervised Speech Representation Learningcited by 1
  14. 2026OLIVE: View-Augmented Latent Prediction with Waveform Reconstruction for Speech SSLcited by 0