mlsanityData to 14 Oct 2026

Discrete Speech Unit Extraction via Independent Component Analysis

arXiv 2501.06562 ↗First posted 11 Jan 2025eess.AS cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Self-supervised speech models (S3Ms) have become a common tool for the speech processing community, leveraging representations for downstream tasks. Clustering S3M representations yields discrete speech units (DSUs), which serve as compact representations for speech signals. DSUs are typically obtained by k-means clustering. Using DSUs often leads to strong performance in various tasks, including automatic speech recognition (ASR). However, even with the high dimensionality and redundancy of S3M representations, preprocessing S3M representations for better clustering remains unexplored, even though it can affect the quality of DSUs. In this paper, we investigate the potential of linear preprocessing methods for extracting DSUs. We evaluate standardization, principal component analysis, whitening, and independent component analysis (ICA) on DSU-based ASR benchmarks and demonstrate their effectiveness as preprocessing for k-means. We also conduct extensive analyses of their behavior, such as orthogonality or interpretability of individual components of ICA.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 38 of its 46 references that are in the map. First posted on arXiv 11 Jan 2025.

Builds on

38 of 46 references.

References · 38

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  4. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  5. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  6. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  7. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
  8. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  9. 2021Generative Spoken Language Modeling from Raw Audiocited by 340
  10. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  11. 2017All-but-the-Top: Simple and Effective Postprocessing for Word Representationscited by 212
  12. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 202
26 more
  1. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
  2. 2021Direct speech-to-speech translation with discrete unitscited by 149
  3. 2021Textless Speech-to-Speech Translation on Real Datacited by 121
  4. 2023Textually Pretrained Speech Language Modelscited by 101
  5. 2023Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speechcited by 101
  6. 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 77
  7. 2023Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation taskscited by 72
  8. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 71
  9. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 71
  10. 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 69
  11. 2024Anisotropy Is Inherent to Self-Attention in Transformerscited by 63
  12. 2021A Comparison of Discrete and Soft Speech Units for Improved Voice Conversioncited by 61
  13. 2022UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Unitscited by 60
  14. 2023Analysing Discrete Self Supervised Speech Representation for Spoken Language Modelingcited by 47
  15. 2023Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Studycited by 42
  16. 2017Simple and Effective Dimensionality Reduction for Word Embeddingscited by 39
  17. 2023Exploration of Efficient End-to-End ASR using Discretized Input from Self-Supervised Learningcited by 28
  18. 2022Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Modelscited by 21
  19. 2023Discovering Universal Geometry in Embeddings with ICAcited by 19
  20. 2022Opening the Black Box of wav2vec Feature Encodercited by 18
  21. 2023An Information-Theoretic Analysis of Self-supervised Discrete Representations of Speechcited by 15
  22. 2014Improved Distributed Principal Component Analysiscited by 14
  23. 2024Orthogonality and isotropy of speaker and phonetic information in self-supervised speech representationscited by 14
  24. 2024The Interspeech 2024 Challenge on Speech Processing Using Discrete Unitscited by 12
  25. 2023Understanding Probe Behaviors through Variational Bounds of Mutual Informationcited by 11
  26. 2024Codec-SUPERB @ SLT 2024: A lightweight benchmark for neural audio codec modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025On-device Streaming Discrete Speech Unitscited by 4
  2. 2025Benchmarking Prosody Encoding in Discrete Speech Tokenscited by 1
  3. 2026Do speech foundation models really learn words?cited by 0