Speech Representation Learning Combining Conformer CPC with Deep Cluster for the ZeroSpeech Challenge 2021
Not on the map. Why?
Abstract (arXiv)
We present a system for the Zero Resource Speech Challenge 2021, which combines a Contrastive Predictive Coding (CPC) with deep cluster. In deep cluster, we first prepare pseudo-labels obtained by clustering the outputs of a CPC network with k-means. Then, we train an additional autoregressive model to classify the previously obtained pseudo-labels in a supervised manner. Phoneme discriminative representation is achieved by executing the second-round clustering with the outputs of the final layer of the autoregressive model. We show that replacing a Transformer layer with a Conformer layer leads to a further gain in a lexical metric. Experimental results show that a relative improvement of 35% in a phonetic metric, 1.5% in the lexical metric, and 2.3% in a syntactic metric are achieved compared to a baseline method of CPC-small which is trained on LibriSpeech 460h data. We achieve top results in this challenge with the syntactic metric.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Jul 2021.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 1; 2022: 4; 2023: 1; 2024: 1.
7 papers.
- 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
- 2022Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challengecited by 34
- 2022Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modelingcited by 25
- 2021DECAR: Deep Clustering for learning general-purpose Audio Representationscited by 7
- 2022Analysis of Self-Supervised Learning and Dimensionality Reduction Methods in Clustering-Based Active Learning for Speech Emotion Recognitioncited by 2
- 2023Representation Learning With Hidden Unit Clustering For Low Resource Speech Applicationscited by 1
- 2024Learning Semantic Information from Raw Audio Signal Using Both Contextual and Phonetic Representationscited by 0
- 2024Learning Semantic Information from Raw Audio Signal Using Both Contextual and Phonetic Representations2 Feb 2024
- 2023Representation Learning With Hidden Unit Clustering For Low Resource Speech Applications14 Jul 2023
- 2022Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challenge27 Oct 2022
- 2022Analysis of Self-Supervised Learning and Dimensionality Reduction Methods in Clustering-Based Active Learning for Speech Emotion Recognition21 Jun 2022
- 2022Self-Supervised Speech Representation Learning: A Review21 May 2022
- 2022Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling7 Feb 2022
- 2021DECAR: Deep Clustering for learning general-purpose Audio Representations17 Oct 2021