# REDAT: Accent-Invariant Representation for End-to-End ASR by Domain Adversarial Training with Relabeling

arXiv: https://arxiv.org/abs/2012.07353 · HTML: https://mlsanity.com/paper/2012.07353/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2020.

## Built on by (most cited first, 18 of 18)

- [Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognition](https://mlsanity.com/paper/2310.06434/) (2023)
- [Improving Distortion Robustness of Self-supervised Speech Processing Tasks with Domain Adaptation](https://mlsanity.com/paper/2203.16104/) (2022)
- [Speech Technology for Everyone: Automatic Speech Recognition for Non-Native English with Transfer Learning](https://mlsanity.com/paper/2110.00678/) (2021)
- [Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models](https://mlsanity.com/paper/2405.14161/) (2024)
- [Improving generalizability of distilled self-supervised speech processing models under distorted settings](https://mlsanity.com/paper/2210.07978/) (2022)
- [Sample-Efficient Unsupervised Domain Adaptation of Speech Recognition Systems A case study for Modern Greek](https://mlsanity.com/paper/2301.00304/) (2022)
- [Accented Speech Recognition With Accent-specific Codebooks](https://mlsanity.com/paper/2310.15970/) (2023)
- [Learning Cross-lingual Visual Speech Representations](https://mlsanity.com/paper/2303.09455/) (2023)
- [Improving Accented Speech Recognition using Data Augmentation based on Unsupervised Text-to-Speech Synthesis](https://mlsanity.com/paper/2407.04047/) (2024)
- [Test-Time Training for Speech](https://mlsanity.com/paper/2309.10930/) (2023)
- [Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding](https://mlsanity.com/paper/2202.05209/) (2022)
- [Unified Modeling of Multi-Domain Multi-Device ASR Systems](https://mlsanity.com/paper/2205.06655/) (2022)
- [Qifusion-Net: Layer-adapted Stream/Non-stream Model for End-to-End Multi-Accent Speech Recognition](https://mlsanity.com/paper/2407.03026/) (2024)
- [Accented Speech Recognition Inspired by Human Perception](https://mlsanity.com/paper/2104.04627/) (2021)
- [MADI: Inter-domain Matching and Intra-domain Discrimination for Cross-domain Speech Recognition](https://mlsanity.com/paper/2302.11224/) (2023)
- [Mixture-of-Experts with Intermediate CTC Supervision for Accented Speech Recognition](https://mlsanity.com/paper/2602.01967/) (2026)
- [A meta learning scheme for fast accent domain expansion in Mandarin speech recognition](https://mlsanity.com/paper/2307.12262/) (2023)
- [Language-Universal Speech Attributes Modeling for Zero-Shot Multilingual Spoken Keyword Recognition](https://mlsanity.com/paper/2406.02488/) (2024)

## Abstract (arXiv)

Accents mismatching is a critical problem for end-to-end ASR. This paper aims to address this problem by building an accent-robust RNN-T system with domain adversarial training (DAT). We unveil the magic behind DAT and provide, for the first time, a theoretical guarantee that DAT learns accent-invariant representations. We also prove that performing the gradient reversal in DAT is equivalent to minimizing the Jensen-Shannon divergence between domain output distributions. Motivated by the proof of equivalence, we introduce reDAT, a novel technique based on DAT, which relabels data using either unsupervised clustering or soft labels. Experiments on 23K hours of multi-accent data show that DAT achieves competitive results over accent-specific baselines on both native and non-native English accents but up to 13% relative WER reduction on unseen accents; our reDAT yields further improvements over DAT by 3% and 8% relatively on non-native accents of American and British English.