mlsanityData to 7 Oct 2026

Towards Matching Phones and Speech Representations

arXiv 2310.17558 ↗First posted 26 Oct 2023cs.CL cs.LG cs.SDcited by 1

Not on the map. Why?

Abstract (arXiv)

Learning phone types from phone instances has been a long-standing problem, while still being open. In this work, we revisit this problem in the context of self-supervised learning, and pose it as the problem of matching cluster centroids to phone embeddings. We study two key properties that enable matching, namely, whether cluster centroids of self-supervised representations reduce the variability of phone instances and respect the relationship among phones. We then use the matching result to produce pseudo-labels and introduce a new loss function for improving self-supervised representations. Our experiments show that the matching result captures the relationship among phones. Training the new loss function jointly with the regular self-supervised losses, such as APC and CPC, significantly improves the downstream phone classification.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 26 Oct 2023.

Builds on

26 of 35 references.

References · 26

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  6. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  7. 2017Word Translation Without Parallel Datacited by 480
  8. 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 185
  9. 2022Self-supervised Learning with Random-projection Quantizer for Speech Recognitioncited by 158
  10. 2021Unsupervised Speech Recognitioncited by 147
  11. 2018Gromov-Wasserstein Alignment of Word Embedding Spacescited by 119
  12. 2021UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Datacited by 76
14 more
  1. 2023Analysing Discrete Self Supervised Speech Representation for Spoken Language Modelingcited by 47
  2. 2017The Zero Resource Speech Challenge 2017cited by 45
  3. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 44
  4. 2020Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentationcited by 43
  5. 2021Joint Unsupervised and Supervised Training for Multilingual ASRcited by 39
  6. 2016A segmental framework for fully-unsupervised large-vocabulary speech recognitioncited by 27
  7. 2018Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddingscited by 22
  8. 2018Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matchingcited by 20
  9. 2023Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspacescited by 19
  10. 2018Unsupervised Hyperalignment for Multilingual Word Embeddingscited by 17
  11. 2021Aligned Contrastive Predictive Codingcited by 15
  12. 2021Unsupervised Speech Segmentation and Variable Rate Representation Learning using Segmental Contrastive Predictive Codingcited by 14
  13. 2022Variable-rate hierarchical CPC leads to acoustic unit discovery in speechcited by 13
  14. 2022Learning Phone Recognition from Unpaired Audio and Phone Sequences Based on Generative Adversarial Networkcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024A Simple HMM with Self-Supervised Representations for Phone Segmentationcited by 1