# Auditory-Based Data Augmentation for End-to-End Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2204.04284 · HTML: https://mlsanity.com/paper/2204.04284/

## Key facts

- Cited by 1 papers in the map (data through 19 Oct 2026).
- Placed in Multilingual and low-resource ASR, one of 11 directions in Speech and audio language models.
- Builds on 12 of its 38 references that are in the map.
- First posted on arXiv 8 Apr 2022.

## Placed in

- Speech and audio language models / [Multilingual and low-resource ASR](https://mlsanity.com/speech/multilingual-and-low-resource-asr/)

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [The fifth 'CHiME' Speech Separation and Recognition Challenge: Dataset, task and baselines](https://mlsanity.com/paper/1803.10609/) (2018)
- [Multi-task self-supervised learning for Robust Speech Recognition](https://mlsanity.com/paper/2001.09239/) (2020)
- [ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context](https://mlsanity.com/paper/2005.03191/) (2020)
- [End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results](https://mlsanity.com/paper/1412.1602/) (2014)
- [Data Augmenting Contrastive Learning of Speech Representations in the Time Domain](https://mlsanity.com/paper/2007.00991/) (2020)
- [Small energy masking for improved neural network training for end-to-end speech recognition](https://mlsanity.com/paper/2002.06312/) (2020)
- [Optimising Hearing Aid Fittings for Speech in Noise with a Differentiable Hearing Loss Model](https://mlsanity.com/paper/2106.04639/) (2021)
- [On the Relevance of Auditory-Based Gabor Features for Deep Learning in Automatic Speech Recognition](https://mlsanity.com/paper/1702.04333/) (2017)

## Built on by (most cited first, 1 of 1)

- [A Policy-based Approach to the SpecAugment Method for Low Resource E2E ASR](https://mlsanity.com/paper/2210.08520/) (2022)

## Abstract (arXiv)

End-to-end models have achieved significant improvement on automatic speech recognition. One common method to improve performance of these models is expanding the data-space through data augmentation. Meanwhile, human auditory inspired front-ends have also demonstrated improvement for automatic speech recognisers. In this work, a well-verified auditory-based model, which can simulate various hearing abilities, is investigated for the purpose of data augmentation for end-to-end speech recognition. By introducing the auditory model into the data augmentation process, end-to-end systems are encouraged to ignore variation from the signal that cannot be heard and thereby focus on robust features for speech recognition. Two mechanisms in the auditory model, spectral smearing and loudness recruitment, are studied on the LibriSpeech dataset with a transformer-based end-to-end model. The results show that the proposed augmentation methods can bring statistically significant improvement on the performance of the state-of-the-art SpecAugment.