Improving Self-supervised Pre-training using Accent-Specific Codebooks
Not on the map. Why?
Abstract (arXiv)
Speech accents present a serious challenge to the performance of state-of-the-art end-to-end Automatic Speech Recognition (ASR) systems. Even with self-supervised learning and pre-training of ASR models, accent invariance is seldom achieved. In this work, we propose an accent-aware adaptation technique for self-supervised learning that introduces a trainable set of accent-specific codebooks to the self-supervised architecture. These learnable codebooks enable the model to capture accent specific information during pre-training, that is further refined during ASR finetuning. On the Mozilla Common Voice dataset, our proposed approach outperforms all other accent-adaptation approaches on both seen and unseen English accents, with up to 9% relative reduction in word error rate (WER).
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 38 references that are in the map. First posted on arXiv 4 Jul 2024.
Builds on
29 of 38 references.
References · 29
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
- 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
- 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
- 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
- 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 340
- 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
- 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
17 more
- 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
- 2022Comparative layer-wise analysis of self-supervised speech modelscited by 155
- 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 148
- 2020DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector Quantizationcited by 77
- 2022SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilitiescited by 76
- 2019Effectiveness of self-supervised pre-training for speech recognitioncited by 72
- 2021Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speechcited by 30
- 2020Learning Fast Adaptation on Cross-Accented Speech Recognitioncited by 28
- 2021Best of Both Worlds: Robust Accented Speech Recognition with Adversarial Transfer Learningcited by 17
- 2021E2E-based Multi-task Learning Approach to Joint Speech and Accent Recognitioncited by 12
- 2021Improving Accent Identification and Accented Speech Recognition Under a Framework of Self-supervised Learningcited by 12
- 2022Layer-wise Fast Adaptation for End-to-End Multi-Accent Speech Recognitioncited by 12
- 2023Accented Speech Recognition With Accent-specific Codebookscited by 6
- 2022CTCBERT: Advancing Hidden-unit BERT with CTC Objectivescited by 5
- 2022Joint Encoder-Decoder Self-Supervised Pre-training for ASRcited by 4
- 2023Don't Stop Self-Supervision: Accent Adaptation of Speech Representations via Residual Adapterscited by 2
- 2023Unsupervised Accent Adaptation Through Masked Language Model Correction Of Discrete Self-Supervised Speech Unitscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.