# PatchDSU: Uncertainty Modeling for Out of Distribution Generalization in Keyword Spotting

arXiv: https://arxiv.org/abs/2508.03190 · HTML: https://mlsanity.com/paper/2508.03190/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 45 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [A Survey of Unsupervised Deep Domain Adaptation](https://mlsanity.com/paper/1812.02849/) (2018)
- [Generalizing Across Domains via Cross-Gradient Training](https://mlsanity.com/paper/1804.10745/) (2018)
- [TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation](https://mlsanity.com/paper/1805.04699/) (2018)
- [Efficient Domain Generalization via Common-Specific Low-Rank Decomposition](https://mlsanity.com/paper/2003.12815/) (2020)
- [Uncertainty Modeling for Out-of-Distribution Generalization](https://mlsanity.com/paper/2202.03958/) (2022)
- [Implicit Semantic Data Augmentation for Deep Networks](https://mlsanity.com/paper/1909.12220/) (2019)
- [Batch-Instance Normalization for Adaptively Style-Invariant Neural Networks](https://mlsanity.com/paper/1805.07925/) (2018)
- [Deep Residual Learning for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1710.10361/) (2017)
- [Temporal Convolution for Real-time Keyword Spotting on Mobile Devices](https://mlsanity.com/paper/1904.03814/) (2019)
- [Broadcasted Residual Learning for Efficient Keyword Spotting](https://mlsanity.com/paper/2106.04140/) (2021)
- [Few-Shot Keyword Spotting in Any Language](https://mlsanity.com/paper/2104.01454/) (2021)
- [Efficient keyword spotting using dilated convolutions and gating](https://mlsanity.com/paper/1811.07684/) (2018)
- [Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting](https://mlsanity.com/paper/2206.15400/) (2022)
- [Domain Generalization with Relaxed Instance Frequency-wise Normalization for Multi-device Acoustic Scene Classification](https://mlsanity.com/paper/2206.12513/) (2022)
- [Zero-shot keyword spotting for visual speech recognition in-the-wild](https://mlsanity.com/paper/1807.08469/) (2018)
- [Small-Footprint Keyword Spotting with Multi-Scale Temporal Convolution](https://mlsanity.com/paper/2010.09960/) (2020)
- [Matching Latent Encoding for Audio-Text based Keyword Spotting](https://mlsanity.com/paper/2306.05245/) (2023)
- [PhonMatchNet: Phoneme-Guided Zero-Shot Keyword Spotting for User-Defined Keywords](https://mlsanity.com/paper/2308.16511/) (2023)
- [SpeechYOLO: Detection and Localization of Speech Objects](https://mlsanity.com/paper/1904.07704/) (2019)

## Abstract (arXiv)

Deep learning models excel at many tasks but rely on the assumption that training and test data follow the same distribution. This assumption often does not hold in real-world speech systems, where distribution shifts are common due to varying environments, recording conditions, and speaker diversity. The method of Domain Shifts with Uncertainty (DSU) augments the input of each neural network layer based on the input feature statistics. It addresses the problem of out-of-domain generalization by assuming feature statistics follow a multivariate Gaussian distribution and substitutes the input with sampled features from this distribution. While effective for computer vision, applying DSU to speech presents challenges due to the nature of the data. Unlike static visual data, speech is a temporal signal commonly represented by a spectrogram - the change of frequency over time. This representation cannot be treated as a simple image, and the resulting sparsity can lead to skewed feature statistics when applied to the entire input. To tackle out-of-distribution issues in keyword spotting, we propose PatchDSU, which extends DSU by splitting the input into patches and independently augmenting each patch. We evaluated PatchDSU and DSU alongside other methods on the Google Speech Commands, Librispeech, and TED-LIUM. Additionally, we evaluated performance under white Gaussian and MUSAN music noise conditions. We also explored out-of-domain generalization by analyzing model performance on datasets they were not trained on. Overall, in most cases, both PatchDSU and DSU outperform other methods. Notably, PatchDSU demonstrates more consistent improvements across the evaluated scenarios compared to other approaches.