mlsanityData to 13 Oct 2026

R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Pieces

arXiv 2311.09117 ↗First posted 15 Nov 2023cs.CL cs.SD eess.AScited by 7

Not on the map. Why?

Abstract (arXiv)

This paper introduces Robust Spin (R-Spin), a data-efficient domain-specific self-supervision method for speaker and noise-invariant speech representations by learning discrete acoustic units with speaker-invariant clustering (Spin). R-Spin resolves Spin's issues and enhances content representations by learning to predict acoustic pieces. R-Spin offers a 12X reduction in computational resources compared to previous state-of-the-art methods while outperforming them in severely distorted speech scenarios. This paper provides detailed analyses to show how discrete units contribute to speech encoder training and improving robustness in diverse acoustic environments.

Key facts. Cited by 7 papers in the map (data through 13 Oct 2026). Builds on 37 of its 53 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

37 of 53 references.

References · 37

  1. 2017Attention Is All You Needcited by 60,106
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,004
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,870
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,910
  6. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,850
  7. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,313
  8. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,826
  9. 2019Similarity of Neural Network Representations Revisitedcited by 1,719
  10. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,625
  11. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,487
  12. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 721
25 more
  1. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 657
  2. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 405
  3. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  4. 2019Speech Model Pre-training for End-to-End Spoken Language Understandingcited by 192
  5. 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 148
  6. 2021DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERTcited by 130
  7. 2021TorchAudio: Building Blocks for Audio and Speech Processingcited by 83
  8. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  9. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 76
  10. 2022Audio Self-supervised Learning: A Surveycited by 65
  11. 2023Analysing Discrete Self Supervised Speech Representation for Spoken Language Modelingcited by 47
  12. 2020Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentationcited by 43
  13. 2023DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learningcited by 34
  14. 2021Segmental Contrastive Predictive Coding for Unsupervised Word Segmentationcited by 32
  15. 2022Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languagescited by 26
  16. 2022Robust Data2vec: Noise-robust Speech Representation Learning for ASR by Combining Regression and Improved Contrastive Learningcited by 21
  17. 2023Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clusteringcited by 21
  18. 2022SPIRAL: Self-supervised Perturbation-Invariant Representation Learning for Speech Pre-Trainingcited by 19
  19. 2022Improving Distortion Robustness of Self-supervised Speech Processing Tasks with Domain Adaptationcited by 16
  20. 2023Acoustic BPE for Speech Generation with Discrete Tokenscited by 15
  21. 2022Speech Pre-training with Acoustic Piececited by 14
  22. 2020Unsupervised Learning of Disentangled Speech Content and Style Representationcited by 13
  23. 2022Phoneme Segmentation Using Self-Supervised Speech Modelscited by 11
  24. 2023Wav2code: Restore Clean Speech Representations via Codebook Lookup for Noise-Robust ASRcited by 10
  25. 2022Content-Context Factorized Representations for Automated Speech Recognitioncited by 8

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 2.

7 papers.

  1. 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 48
  2. 2024Improving Spoken Language Modeling with Phoneme Classification: A Simple Fine-tuning Approachcited by 8
  3. 2024DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Modelscited by 8
  4. 2025SpidR: Learning Fast and Stable Linguistic Units for Spoken Language Models Without Supervisioncited by 6
  5. 2025StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMscited by 4
  6. 2026Position-invariant Fine-tuning of Speech Enhancement Models with Self-supervised Speech Representationscited by 0
  7. 2026SISER: Speaker-Invariant Speech Emotion Recognition with Entropy-Based Adversarial Trainingcited by 0