mlsanityData to 14 Oct 2026

MixRep: Hidden Representation Mixup for Low-Resource Speech Recognition

arXiv 2310.18450 ↗First posted 27 Oct 2023eess.AS cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

In this paper, we present MixRep, a simple and effective data augmentation strategy based on mixup for low-resource ASR. MixRep interpolates the feature dimensions of hidden representations in the neural network that can be applied to both the acoustic feature input and the output of each layer, which generalizes the previous MixSpeech method. Further, we propose to combine the mixup with a regularization along the time axis of the input, which is shown as complementary. We apply MixRep to a Conformer encoder of an E2E LAS architecture trained with a joint CTC loss. We experiment on the WSJ dataset and subsets of the SWB dataset, covering reading and telephony conversational speech. Experimental results show that MixRep consistently outperforms other regularization methods for low-resource ASR. Compared to a strong SpecAugment baseline, MixRep achieves a +6.5% and a +6.7% relative WER reduction on the eval92 set and the Callhome part of the eval'2000 set.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 25 of its 45 references that are in the map. First posted on arXiv 27 Oct 2023.

Builds on

25 of 45 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  4. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  5. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  6. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
  7. 2015Listen, Attend and Spellcited by 543
  8. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 257
  9. 2015End-to-End Attention-based Large Vocabulary Speech Recognitioncited by 151
  10. 2021Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Trainingcited by 151
  11. 2017Multilingual Speech Recognition With A Single End-To-End Modelcited by 87
  12. 2020Sequence-Level Mixed Sample Data Augmentationcited by 71
13 more
  1. 2020Imputer: Sequence Modelling via Imputation and Dynamic Programmingcited by 66
  2. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 64
  3. 2022STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translationcited by 57
  4. 2020On the Generalization Effects of Linear Transformations in Data Augmentationcited by 47
  5. 2021MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognitioncited by 39
  6. 2017Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systemscited by 32
  7. 2021On the limit of English conversational speech recognitioncited by 30
  8. 2020Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation from a Blackbox Modelcited by 28
  9. 2020Improving Low Resource Code-switched ASR using Augmented Code-switched TTScited by 15
  10. 2019What does a network layer hear? Analyzing hidden representations of end-to-end ASR through speech synthesiscited by 13
  11. 2013Improvements to deep convolutional neural networks for LVCSRcited by 11
  12. 2021Data Augmentation Methods for End-to-end Speech Recognition on Distant-Talk Scenarioscited by 6
  13. 2021Scenario Aware Speech Recognition: Advancements for Apollo Fearless Steps & CHiME-4 Corporacited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Learn and Don't Forget: Adding a New Language to ASR Foundation Modelscited by 16
  2. 2025Bridging the Language Gap: Synthetic Voice Diversity via Latent Mixup for Equitable Speech Recognitioncited by 0