mlsanityData to 7 Oct 2026

Segmental Contrastive Predictive Coding for Unsupervised Word Segmentation

arXiv 2106.02170 ↗First posted 3 Jun 2021eess.AS cs.CL cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Automatic detection of phoneme or word-like units is one of the core objectives in zero-resource speech processing. Recent attempts employ self-supervised training methods, such as contrastive predictive coding (CPC), where the next frame is predicted given past context. However, CPC only looks at the audio signal's frame-level structure. We overcome this limitation with a segmental contrastive predictive coding (SCPC) framework that can model the signal structure at a higher level e.g. at the phoneme level. In this framework, a convolutional neural network learns frame-level representation from the raw waveform via noise-contrastive estimation (NCE). A differentiable boundary detector finds variable-length segments, which are then used to optimize a segment encoder via NCE to learn segment representations. The differentiable boundary detector allows us to train frame-level and segment-level encoders jointly. Typically, phoneme and word segmentation are treated as separate tasks. We unify them and experimentally show that our single model outperforms existing phoneme and word segmentation methods on TIMIT and Buckeye datasets. We analyze the impact of boundary threshold and when is the right time to include the segmental loss in the learning process.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 14 of its 28 references that are in the map. First posted on arXiv 3 Jun 2021.

Builds on

14 of 28 references.

References · 14

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  5. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
  6. 2019Unsupervised speech representation learning using WaveNet autoencoderscited by 90
  7. 2020Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challengecited by 55
  8. 2020Self-Supervised Contrastive Learning for Unsupervised Phoneme Segmentationcited by 43
  9. 2019The Zero Resource Speech Challenge 2019: TTS without Tcited by 38
  10. 2017An embedded segmental K-means model for unsupervised segmentation and clustering of speechcited by 31
  11. 2016A segmental framework for fully-unsupervised large-vocabulary speech recognitioncited by 27
  12. 2020Towards unsupervised phone and word segmentation using self-supervised vector-quantized neural networkscited by 24
2 more
  1. 2016Blind phoneme segmentation with temporal prediction errorscited by 9
  2. 2017Gate Activation Signal Analysis for Gated Recurrent Neural Networks and Its Correlation with Phoneme Boundariescited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 13; 2023: 8; 2024: 4; 2025: 3; 2026: 2.

32 papers.

Sort:

By area:

  1. 2022Efficient Transformers with Dynamic Token Poolingcited by 75
  2. 2023What Do Self-Supervised Speech Models Know About Words?cited by 62
  3. 2022Word Discovery in Visually Grounded, Self-Supervised Speech Modelscited by 37
  4. 2022Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challengecited by 34
  5. 2021Phone-to-audio alignment without text: A Semi-supervised Approachcited by 25
  6. 2022Word Segmentation on Discovered Phone Units with Dynamic Programming and Self-Supervised Scoringcited by 24
  7. 2021Contrastive prediction strategies for unsupervised segmentation and categorization of phonemes and wordscited by 18
  8. 2022Variable-rate hierarchical CPC leads to acoustic unit discovery in speechcited by 13
  9. 2022DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexiconcited by 12
  10. 2022Evaluating context-invariance in unsupervised speech representationscited by 11
  11. 2022Phoneme Segmentation Using Self-Supervised Speech Modelscited by 11
  12. 2023Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Modelcited by 11
20 more
  1. 2022A Brief Overview of Unsupervised Neural Speech Representation Learningcited by 9
  2. 2022Autoregressive Co-Training for Learning Discrete Speech Representationscited by 8
  3. 2022Speech Sequence Embeddings using Nearest Neighbors Contrastive Learningcited by 7
  4. 2022On Compressing Sequences for Self-Supervised Speech Modelscited by 7
  5. 2023Unsupervised Word Segmentation Using Temporal Gradient Pseudo-Labelscited by 7
  6. 2023R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Piecescited by 7
  7. 2024Unsupervised Word Discovery: Boundary Detection with Clustering vs. Dynamic Programmingcited by 6
  8. 2023Unsupervised Speech Recognition with N-Skipgram and Positional Unigram Matchingcited by 5
  9. 2025Should Top-Down Clustering Affect Boundaries in Unsupervised Word Discovery?cited by 4
  10. 2025Unsupervised lexicon learning from speech is limited by representations rather than clusteringcited by 4
  11. 2025Towards Audio Token Compression in Large Audio Language Modelscited by 4
  12. 2024REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASRcited by 3
  13. 2023Cascading and Direct Approaches to Unsupervised Constituency Parsing on Spoken Sentencescited by 2
  14. 2023Audio-Visual Neural Syntax Acquisitioncited by 2
  15. 2022Learning Dependencies of Discrete Speech Representations with Neural Hidden Markov Modelscited by 1
  16. 2023XLS-R fine-tuning on noisy word boundaries for unsupervised speech segmentation into wordscited by 1
  17. 2024Removing Speaker Information from Speech Representation using Variable-Length Soft Poolingcited by 1
  18. 2024Smooth InfoMax -- Towards Easier Post-Hoc Interpretabilitycited by 0
  19. 2026You Can Learn Tokenization End-to-End with Reinforcement Learningcited by 0
  20. 2026Recovering the Zipfian Distribution in Unsupervised Term Discoverycited by 0