# MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2102.12664 · HTML: https://mlsanity.com/paper/2102.12664/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 29 references that are in the map.
- First posted on arXiv 25 Feb 2021.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [Augmenting Data with Mixup for Sentence Classification: An Empirical Study](https://mlsanity.com/paper/1905.08941/) (2019)
- [Transformers with convolutional context for ASR](https://mlsanity.com/paper/1904.11660/) (2019)
- [LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition](https://mlsanity.com/paper/2008.03687/) (2020)
- [Almost Unsupervised Text to Speech and Automatic Speech Recognition](https://mlsanity.com/paper/1905.06791/) (2019)
- [Semantic Mask for Transformer based End-to-End Speech Recognition](https://mlsanity.com/paper/1912.03010/) (2019)
- [Deep Convolutional Neural Network with Mixup for Environmental Sound Classification](https://mlsanity.com/paper/1808.08405/) (2018)

## Built on by (most cited first, 30 of 39)

- [End-to-End Speech Recognition: A Survey](https://mlsanity.com/paper/2303.03329/) (2023)
- [STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation](https://mlsanity.com/paper/2203.10426/) (2022)
- [A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainability](https://mlsanity.com/paper/2212.10888/) (2022)
- [A Unified Analysis of Mixed Sample Data Augmentation: A Loss Function Perspective](https://mlsanity.com/paper/2208.09913/) (2022)
- [M3ST: Mix at Three Levels for Speech Translation](https://mlsanity.com/paper/2212.03657/) (2022)
- [Cross-modality Data Augmentation for End-to-End Sign Language Translation](https://mlsanity.com/paper/2305.11096/) (2023)
- [CCC-wav2vec 2.0: Clustering aided Cross Contrastive Self-supervised learning of speech representations](https://mlsanity.com/paper/2210.02592/) (2022)
- [Exploring Temporally Dynamic Data Augmentation for Video Recognition](https://mlsanity.com/paper/2206.15015/) (2022)
- [R-Mixup: Riemannian Mixup for Biological Networks](https://mlsanity.com/paper/2306.02532/) (2023)
- [Posterior Uncertainty Quantification in Neural Networks using Data Augmentation](https://mlsanity.com/paper/2403.12729/) (2024)
- [Selective Mixup Helps with Distribution Shifts, But Not (Only) because of Mixup](https://mlsanity.com/paper/2305.16817/) (2023)
- [SpliceOut: A Simple and Efficient Audio Augmentation Method](https://mlsanity.com/paper/2110.00046/) (2021)
- [Make More of Your Data: Minimal Effort Data Augmentation for Automatic Speech Recognition and Translation](https://mlsanity.com/paper/2210.15398/) (2022)
- [Weighted Cross-entropy for Low-Resource Languages in Multilingual Speech Recognition](https://mlsanity.com/paper/2409.16954/) (2024)
- [Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning](https://mlsanity.com/paper/2302.11343/) (2023)
- [Tailoring Mixup to Data for Calibration](https://mlsanity.com/paper/2311.01434/) (2023)
- [Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languages](https://mlsanity.com/paper/2503.23542/) (2025)
- [CoSDA: Continual Source-Free Domain Adaptation](https://mlsanity.com/paper/2304.06627/) (2023)
- [Learning Emotional Representations from Imbalanced Speech Data for Speech Emotion Recognition and Emotional Text-to-Speech](https://mlsanity.com/paper/2306.05709/) (2023)
- [IntraMix: Intra-Class Mixup Generation for Accurate Labels and Neighbors](https://mlsanity.com/paper/2405.00957/) (2024)
- [PM-MMUT: Boosted Phone-Mask Data Augmentation using Multi-Modeling Unit Training for Phonetic-Reduction-Robust E2E Speech Recognition](https://mlsanity.com/paper/2112.06721/) (2021)
- [SingAug: Data Augmentation for Singing Voice Synthesis with Cycle-consistent Training Strategy](https://mlsanity.com/paper/2203.17001/) (2022)
- [Language-Universal Phonetic Representation in Multilingual Speech Pretraining for Low-Resource Speech Recognition](https://mlsanity.com/paper/2305.11569/) (2023)
- [Empirical Study of Mix-based Data Augmentation Methods in Physiological Time Series Data](https://mlsanity.com/paper/2309.09970/) (2023)
- [Hate Speech Detection in Limited Data Contexts using Synthetic Data Generation](https://mlsanity.com/paper/2310.02876/) (2023)
- [Matching-based Term Semantics Pre-training for Spoken Patient Query Understanding](https://mlsanity.com/paper/2303.01341/) (2023)
- [Language-universal phonetic encoder for low-resource speech recognition](https://mlsanity.com/paper/2305.11576/) (2023)
- [MixRep: Hidden Representation Mixup for Low-Resource Speech Recognition](https://mlsanity.com/paper/2310.18450/) (2023)
- [NeuroAMP: A Novel End-to-end General Purpose Deep Neural Amplifier for Personalized Hearing Aids](https://mlsanity.com/paper/2502.10822/) (2025)
- [Scaling Ambiguity: Augmenting Human Annotation in Speech Emotion Recognition with Audio-Language Models](https://mlsanity.com/paper/2601.14620/) (2026)

## Abstract (arXiv)

In this paper, we propose MixSpeech, a simple yet effective data augmentation method based on mixup for automatic speech recognition (ASR). MixSpeech trains an ASR model by taking a weighted combination of two different speech features (e.g., mel-spectrograms or MFCC) as the input, and recognizing both text sequences, where the two recognition losses use the same combination weight. We apply MixSpeech on two popular end-to-end speech recognition models including LAS (Listen, Attend and Spell) and Transformer, and conduct experiments on several low-resource datasets including TIMIT, WSJ, and HKUST. Experimental results show that MixSpeech achieves better accuracy than the baseline models without data augmentation, and outperforms a strong data augmentation method SpecAugment on these recognition tasks. Specifically, MixSpeech outperforms SpecAugment with a relative PER improvement of 10.6% on TIMIT dataset, and achieves a strong WER of 4.7% on WSJ dataset.