# Towards Robust Waveform-Based Acoustic Models

arXiv: https://arxiv.org/abs/2110.08634 · HTML: https://mlsanity.com/paper/2110.08634/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 53 references that are in the map.
- First posted on arXiv 16 Oct 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers](https://mlsanity.com/paper/1906.04584/) (2019)
- [Regularisation of Neural Networks by Enforcing Lipschitz Continuity](https://mlsanity.com/paper/1804.04368/) (2018)
- [Pyroomacoustics: A Python package for audio room simulations and array processing algorithms](https://mlsanity.com/paper/1710.04196/) (2017)
- [A Kernel Theory of Modern Data Augmentation](https://mlsanity.com/paper/1803.06084/) (2018)
- [You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentation](https://mlsanity.com/paper/2005.07157/) (2020)
- [Multi-Modal Data Augmentation for End-to-End ASR](https://mlsanity.com/paper/1803.10299/) (2018)
- [SMS-WSJ: Database, performance measures, and baseline recipe for multi-channel source separation and recognition](https://mlsanity.com/paper/1910.13934/) (2019)
- [Speech and Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1812.05920/) (2018)
- [Learning Waveform-Based Acoustic Models using Deep Variational Convolutional Neural Networks](https://mlsanity.com/paper/1906.09526/) (2019)
- [Multi-scale Octave Convolutions for Robust Speech Recognition](https://mlsanity.com/paper/1910.14443/) (2019)
- [Frame-level SpecAugment for Deep Convolutional Neural Networks in Hybrid ASR Systems](https://mlsanity.com/paper/2012.04094/) (2020)

## Built on by (most cited first, 1 of 1)

- [Shortcut Learning in Binary Classifier Black Boxes: Applications to Voice Anti-Spoofing and Biometrics](https://mlsanity.com/paper/2601.17782/) (2026)

## Abstract (arXiv)

We study the problem of learning robust acoustic models in adverse environments, characterized by a significant mismatch between training and test conditions. This problem is of paramount importance for the deployment of speech recognition systems that need to perform well in unseen environments. First, we characterize data augmentation theoretically as an instance of vicinal risk minimization, which aims at improving risk estimates during training by replacing the delta functions that define the empirical density over the input space with an approximation of the marginal population density in the vicinity of the training samples. More specifically, we assume that local neighborhoods centered at training samples can be approximated using a mixture of Gaussians, and demonstrate theoretically that this can incorporate robust inductive bias into the learning process. We then specify the individual mixture components implicitly via data augmentation schemes, designed to address common sources of spurious correlations in acoustic models. To avoid potential confounding effects on robustness due to information loss, which has been associated with standard feature extraction techniques (e.g., FBANK and MFCC features), we focus on the waveform-based setting. Our empirical results show that the approach can generalize to unseen noise conditions, with 150% relative improvement in out-of-distribution generalization compared to training using the standard risk minimization principle. Moreover, the results demonstrate competitive performance relative to models learned using a training sample designed to match the acoustic conditions characteristic of test utterances.