mlsanityData to 7 Oct 2026

Speech Sequence Embeddings using Nearest Neighbors Contrastive Learning

arXiv 2204.05148 ↗First posted 11 Apr 2022cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

We introduce a simple neural encoder architecture that can be trained using an unsupervised contrastive learning objective which gets its positive samples from data-augmented k-Nearest Neighbors search. We show that when built on top of recent self-supervised audio representations, this method can be applied iteratively and yield competitive SSE as evaluated on two tasks: query-by-example of random sequences of speech, and spoken term discovery. On both tasks our method pushes the state-of-the-art by a significant margin across 5 different languages. Finally, we establish a benchmark on a query-by-example task on the LibriSpeech dataset to monitor future improvements in the field.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 28 of its 43 references that are in the map. First posted on arXiv 11 Apr 2022.

Builds on

28 of 43 references.

References · 28

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2017Billion-scale similarity search with GPUscited by 2,477
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  6. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  7. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 324
  8. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  9. 2021Unsupervised Speech Recognitioncited by 147
  10. 2020Unsupervised pretraining transfers well across languagescited by 100
  11. 2020The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modelingcited by 100
  12. 2020Data Augmenting Contrastive Learning of Speech Representations in the Time Domaincited by 63
16 more
  1. 2020PyTorch Metric Learningcited by 57
  2. 2020The Zero Resource Speech Challenge 2020: Discovering discrete subword and word unitscited by 39
  3. 2016Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencodercited by 34
  4. 2018Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spacescited by 34
  5. 2019rVAD: An Unsupervised Segment-Based Robust Voice Activity Detection Methodcited by 32
  6. 2021Segmental Contrastive Predictive Coding for Unsupervised Word Segmentationcited by 32
  7. 2017An embedded segmental K-means model for unsupervised segmentation and clustering of speechcited by 31
  8. 2016Discriminative Acoustic Word Embeddings: Recurrent Neural Network-Based Approachescited by 25
  9. 2020Evaluating the reliability of acoustic speech embeddingscited by 21
  10. 2021Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptationcited by 19
  11. 2020Unsupervised Discovery of Recurring Speech Patterns Using Probabilistic Adaptive Metricscited by 16
  12. 2020Self-Expressing Autoencoders for Unsupervised Spoken Term Discoverycited by 12
  13. 2020A comparison of self-supervised speech representations as input features for unsupervised acoustic word embeddingscited by 10
  14. 2021How Familiar Does That Sound? Cross-Lingual Representational Similarity Analysis of Acoustic Word Embeddingscited by 8
  15. 2018Multilingual and Unsupervised Subword Modeling for Zero-Resource Languagescited by 6
  16. 2020Unsupervised feature learning for speech using correspondence and Siamese networkscited by 3

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3; 2024: 1; 2025: 1.

7 papers.

  1. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 116
  2. 2024Recent Advances in Speech Language Models: A Surveycited by 85
  3. 2022Analyzing Acoustic Word Embeddings from Pre-trained Self-supervised Speech Modelscited by 21
  4. 2023Generative Spoken Language Model based on continuous word-sized audio tokenscited by 19
  5. 2022DP-Parse: Finding Word Boundaries from Raw Speech with an Instance Lexiconcited by 12
  6. 2023Acoustic Word Embeddings for Untranscribed Target Languages with Continued Pretraining and Learned Poolingcited by 2
  7. 2023XLS-R fine-tuning on noisy word boundaries for unsupervised speech segmentation into wordscited by 1