mlsanityData to 7 Oct 2026

Data Augmenting Contrastive Learning of Speech Representations in the Time Domain

arXiv 2007.00991 ↗First posted 2 Jul 2020eess.AS cs.CL cs.SDcited by 63

Not on the map. Why?

Abstract (arXiv)

Contrastive Predictive Coding (CPC), based on predicting future segments of speech based on past segments is emerging as a powerful algorithm for representation learning of speech signal. However, it still under-performs other methods on unsupervised evaluation benchmarks. Here, we introduce WavAugment, a time-domain data augmentation library and find that applying augmentation in the past is generally more efficient and yields better performances than other methods. We find that a combination of pitch modification, additive noise and reverberation substantially increase the performance of CPC (relative improvement of 18-22%), beating the reference Libri-light results with 600 times less data. Using an out-of-domain dataset, time-domain data augmentation can push CPC to be on par with the state of the art on the Zero Speech Benchmark 2017. We also show that time-domain data augmentation consistently improves downstream limited-supervision phoneme classification tasks by a factor of 12-15% relative.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 21; 2023: 10; 2024: 11; 2025: 3; 2026: 2.

63 papers.

Sort:

By area:

  1. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  2. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  3. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  4. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  5. 2022Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Predictioncited by 277
  6. 2021End-to-end Audio-visual Speech Recognition with Conformerscited by 147
  7. 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
  8. 2021Contrastive Learning of Musical Representationscited by 81
  9. 2022ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakerscited by 73
  10. 2024Best-of-N Jailbreakingcited by 71
  11. 2022Audio Self-supervised Learning: A Surveycited by 65
  12. 2021Multi-Format Contrastive Learning of Audio Representationscited by 39
38 more
  1. 2021Self-supervised Contrastive Cross-Modality Representation Learning for Spoken Question Answeringcited by 39
  2. 2022AugLy: Data Augmentations for Robustnesscited by 34
  3. 2022Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challengecited by 34
  4. 2021Textless Speech Emotion Conversion using Discrete and Decomposed Representationscited by 29
  5. 2022textless-lib: a Library for Textless Spoken Language Processingcited by 28
  6. 2025Learning Transformer-based World Models with Contrastive Predictive Codingcited by 20
  7. 2022Augmentation Invariant Discrete Representation for Generative Spoken Language Modelingcited by 17
  8. 2021End-to-End Speech Translation with Pre-trained Models and Adapters: UPC at IWSLT 2021cited by 16
  9. 2021Self-Supervised Learning for speech recognition with Intermediate layer supervisioncited by 15
  10. 2022CCC-wav2vec 2.0: Clustering aided Cross Contrastive Self-supervised learning of speech representationscited by 11
  11. 2021Supervised Contrastive Learning for Accented Speech Recognitioncited by 10
  12. 2021From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehensioncited by 10
  13. 2022Self-supervised Context-aware Style Representation for Expressive Speech Synthesiscited by 10
  14. 2022Wav2Vec-Aug: Improved self-supervised training with limited datacited by 9
  15. 2022Feature Dropout: Revisiting the Role of Augmentations in Contrastive Learningcited by 9
  16. 2021Multi-view Contrastive Self-Supervised Learning of Accounting Data Representations for Downstream Audit Taskscited by 8
  17. 2022Speech Sequence Embeddings using Nearest Neighbors Contrastive Learningcited by 7
  18. 2022Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processingcited by 7
  19. 2021Zero-shot Voice Conversion via Self-supervised Prosody Representation Learningcited by 6
  20. 2023Tagged End-to-End Simultaneous Speech Translation Training using Simultaneous Interpretation Datacited by 6
  21. 2022Towards Proper Contrastive Self-supervised Learning Strategies For Music Audio Representationcited by 5
  22. 2022Improving generalizability of distilled self-supervised speech processing models under distorted settingscited by 5
  23. 2021NoiseVC: Towards High Quality Zero-Shot Voice Conversioncited by 4
  24. 2022Automatic Data Augmentation Selection and Parametrization in Contrastive Self-Supervised Speech Representation Learningcited by 4
  25. 2022SLICER: Learning universal audio representations using low-resource self-supervised pre-trainingcited by 4
  26. 2024On the Effect of Data-Augmentation on Local Embedding Properties in the Contrastive Learning of Music Audio Representationscited by 4
  27. 2023Inter-connection: Effective Connection between Pre-trained Encoder and Decoder for Speech Translationcited by 3
  28. 2024Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Paircited by 3
  29. 2025Delayed Bidirectional Alignment via Disentangled Audio Semantics for Audio-Visual Segmentationcited by 3
  30. 2023Discovering COVID-19 Coughing and Breathing Patterns from Unlabeled Data Using Contrastive Learning with Varying Pre-Training Domainscited by 2
  31. 2024MAD Speech: Measures of Acoustic Diversity of Speechcited by 2
  32. 2024Audio Explanation Synthesis with Generative Foundation Modelscited by 2
  33. 2022Exploiting Single-Channel Speech for Multi-Channel End-to-End Speech Recognition: A Comparative Studycited by 1
  34. 2022Speech Augmentation Based Unsupervised Learning for Keyword Spottingcited by 1
  35. 2022DDKtor: Automatic Diadochokinetic Speech Analysiscited by 1
  36. 2023MADI: Inter-domain Matching and Intra-domain Discrimination for Cross-domain Speech Recognitioncited by 1
  37. 2023Speech Translation with Foundation Models and Optimal Transport: UPC at IWSLT23cited by 1
  38. 2023Robust Automatic Speech Recognition via WavAugment Guided Phoneme Adversarial Trainingcited by 1

50 most cited of 63 All 63 (JSON)