# Instabilities in Convnets for Raw Audio

arXiv: https://arxiv.org/abs/2309.05855 · HTML: https://mlsanity.com/paper/2309.05855/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 36 references that are in the map.
- First posted on arXiv 11 Sep 2023.

## Builds on (most cited first, 10 of 10)

- [Lipschitz regularity of deep neural networks: analysis and efficient estimation](https://mlsanity.com/paper/1805.10965/) (2018)
- [Parseval Networks: Improving Robustness to Adversarial Examples](https://mlsanity.com/paper/1704.08847/) (2017)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [Computational bioacoustics with deep learning: a review and roadmap](https://mlsanity.com/paper/2112.06725/) (2021)
- [LEAF: A Learnable Frontend for Audio Classification](https://mlsanity.com/paper/2101.08596/) (2021)
- [Learning Filterbanks from Raw Speech for Phone Recognition](https://mlsanity.com/paper/1711.01161/) (2017)
- [EfficientLEAF: A Faster LEarnable Audio Frontend of Questionable Use](https://mlsanity.com/paper/2207.05508/) (2022)
- [Exploring Filterbank Learning for Keyword Spotting](https://mlsanity.com/paper/2006.00217/) (2020)
- [Fitting Auditory Filterbanks with Multiresolution Neural Networks](https://mlsanity.com/paper/2307.13821/) (2023)
- [Frame Theory for Signal Processing in Psychoacoustics](https://mlsanity.com/paper/1611.00966/) (2016)

## Built on by (most cited first, 2 of 2)

- [Hold Me Tight: Stable Encoder-Decoder Design for Speech Enhancement](https://mlsanity.com/paper/2408.17358/) (2024)
- [ISAC: An Invertible and Stable Auditory Filter Bank with Customizable Kernels for ML Integration](https://mlsanity.com/paper/2505.07709/) (2025)

## Abstract (arXiv)

What makes waveform-based deep learning so hard? Despite numerous attempts at training convolutional neural networks (convnets) for filterbank design, they often fail to outperform hand-crafted baselines. These baselines are linear time-invariant systems: as such, they can be approximated by convnets with wide receptive fields. Yet, in practice, gradient-based optimization leads to suboptimal approximations. In our article, we approach this phenomenon from the perspective of initialization. We present a theory of large deviations for the energy response of FIR filterbanks with random Gaussian weights. We find that deviations worsen for large filters and locally periodic input signals, which are both typical for audio signal processing applications. Numerical simulations align with our theory and suggest that the condition number of a convolutional layer follows a logarithmic scaling law between the number and length of the filters, which is reminiscent of discrete wavelet bases.