Improving Speech Representation Learning via Speech-level and Phoneme-level Masking Approach
Not on the map. Why?
Abstract (arXiv)
Recovering the masked speech frames is widely applied in speech representation learning. However, most of these models use random masking in the pre-training. In this work, we proposed two kinds of masking approaches: (1) speech-level masking, making the model to mask more speech segments than silence segments, (2) phoneme-level masking, forcing the model to mask the whole frames of the phoneme, instead of phoneme pieces. We pre-trained the model via these two approaches, and evaluated on two downstream tasks, phoneme classification and speaker recognition. The experiments demonstrated that the proposed masking approaches are beneficial to improve the performance of speech representation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 24 references that are in the map. First posted on arXiv 25 Oct 2022.
Builds on
18 of 24 references.
References · 18
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
- 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
- 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
- 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 196
- 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 186
- 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 185
- 2020Unsupervised Pre-training of Bidirectional Speech Encoders via Masked Reconstructioncited by 38
6 more
- 2016Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencodercited by 34
- 2019Improving Transformer-based Speech Recognition Using Unsupervised Pre-trainingcited by 29
- 2021Wav2vec-C: A Self-supervised Model for Speech Representation Learningcited by 25
- 2022AVQVC: One-shot Voice Conversion by Vector Quantization with applying contrastive learningcited by 24
- 2020Large-scale Transfer Learning for Low-resource Spoken Language Understandingcited by 11
- 2022TDASS: Target Domain Adaptation Speech Synthesis Framework for Multi-speaker Low-Resource TTScited by 5
Built on by
Citing papers per year: 2025: 1.
1 paper.