mlsanityData to 7 Oct 2026

Contrastive prediction strategies for unsupervised segmentation and categorization of phonemes and words

arXiv 2110.15909 ↗First posted 29 Oct 2021cs.LG cs.SD eess.AScited by 18

Not on the map. Why?

Abstract (arXiv)

We investigate the performance on phoneme categorization and phoneme and word segmentation of several self-supervised learning (SSL) methods based on Contrastive Predictive Coding (CPC). Our experiments show that with the existing algorithms there is a trade off between categorization and segmentation performance. We investigate the source of this conflict and conclude that the use of context building networks, albeit necessary for superior performance on categorization tasks, harms segmentation performance by causing a temporal shift on the learned representations. Aiming to bridge this gap, we take inspiration from the leading approach on segmentation, which simultaneously models the speech signal at the frame and phoneme level, and incorporate multi-level modelling into Aligned CPC (ACPC), a variation of CPC which exhibits the best performance on categorization tasks. Our multi-level ACPC (mACPC) improves in all categorization metrics and achieves state-of-the-art performance in word segmentation.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 7 of its 14 references that are in the map. First posted on arXiv 29 Oct 2021.

Builds on

7 of 14 references.

References · 7

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2020The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modelingcited by 100
  4. 2020Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentationcited by 43
  5. 2021Segmental Contrastive Predictive Coding for Unsupervised Word Segmentationcited by 32
  6. 2020Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networkscited by 24
  7. 2021Aligned Contrastive Predictive Codingcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 4; 2024: 3; 2025: 3; 2026: 1.

18 papers.

Sort:

By area:

  1. 2023What Do Self-Supervised Speech Models Know About Words?cited by 62
  2. 2022Word Discovery in Visually Grounded, Self-Supervised Speech Modelscited by 37
  3. 2022Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challengecited by 34
  4. 2022Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoringcited by 24
  5. 2022Variable-rate hierarchical CPC leads to acoustic unit discovery in speechcited by 13
  6. 2022DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexiconcited by 12
  7. 2023Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Modelcited by 11
  8. 2022A Brief Overview of Unsupervised Neural Speech Representation Learningcited by 9
  9. 2023Unsupervised Word Segmentation Using Temporal Gradient Pseudo-Labelscited by 7
  10. 2024Unsupervised Word Discovery: Boundary Detection with Clustering vs. Dynamic Programmingcited by 6
  11. 2022Unsupervised Symbolic Music Segmentation using Ensemble Temporal Prediction Errorscited by 4
  12. 2025Should Top-Down Clustering Affect Boundaries in Unsupervised Word Discovery?cited by 4
6 more
  1. 2025Unsupervised lexicon learning from speech is limited by representations rather than clusteringcited by 4
  2. 2025Towards Audio Token Compression in Large Audio Language Modelscited by 4
  3. 2023Leveraging multilingual transfer for unsupervised semantic acoustic word embeddingscited by 2
  4. 2024Towards Unsupervised Speech Recognition Without Pronunciation Modelscited by 1
  5. 2024A Simple HMM with Self-Supervised Representations for Phone Segmentationcited by 1
  6. 2026Recovering the Zipfian Distribution in Unsupervised Term Discoverycited by 0