# Fitting Auditory Filterbanks with Multiresolution Neural Networks

arXiv: https://arxiv.org/abs/2307.13821 · HTML: https://mlsanity.com/paper/2307.13821/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 38 references that are in the map.
- First posted on arXiv 25 Jul 2023.

## Builds on (most cited first, 12 of 12)

- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Parallel WaveNet: Fast High-Fidelity Speech Synthesis](https://mlsanity.com/paper/1711.10433/) (2017)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [Computational bioacoustics with deep learning: a review and roadmap](https://mlsanity.com/paper/2112.06725/) (2021)
- [LEAF: A Learnable Frontend for Audio Classification](https://mlsanity.com/paper/2101.08596/) (2021)
- [Learning Filterbanks from Raw Speech for Phone Recognition](https://mlsanity.com/paper/1711.01161/) (2017)
- [End-to-end music source separation: is it possible in the waveform domain?](https://mlsanity.com/paper/1810.12187/) (2018)
- [Deep Learning in Cardiology](https://mlsanity.com/paper/1902.11122/) (2019)
- [EfficientLEAF: A Faster LEarnable Audio Frontend of Questionable Use](https://mlsanity.com/paper/2207.05508/) (2022)
- [OrchideaSOL: a dataset of extended instrumental techniques for computer-aided orchestration](https://mlsanity.com/paper/2007.00763/) (2020)
- [Learnable Frontends that do not Learn: Quantifying Sensitivity to Filterbank Initialisation](https://mlsanity.com/paper/2302.10014/) (2023)
- [Adaptive Representations of Sound for Automatic Insect Recognition](https://mlsanity.com/paper/2304.12739/) (2023)

## Built on by (most cited first, 4 of 4)

- [Instabilities in Convnets for Raw Audio](https://mlsanity.com/paper/2309.05855/) (2023)
- [Hold Me Tight: Stable Encoder-Decoder Design for Speech Enhancement](https://mlsanity.com/paper/2408.17358/) (2024)
- [Aliasing in Convnets: A Frame-Theoretic Perspective](https://mlsanity.com/paper/2507.06152/) (2025)
- [AaSP: Aliasing-aware Self-Supervised Pre-Training for Audio Spectrogram Transformers](https://mlsanity.com/paper/2512.03637/) (2025)

## Abstract (arXiv)

Waveform-based deep learning faces a dilemma between nonparametric and parametric approaches. On one hand, convolutional neural networks (convnets) may approximate any linear time-invariant system; yet, in practice, their frequency responses become more irregular as their receptive fields grow. On the other hand, a parametric model such as LEAF is guaranteed to yield Gabor filters, hence an optimal time-frequency localization; yet, this strong inductive bias comes at the detriment of representational capacity. In this paper, we aim to overcome this dilemma by introducing a neural audio model, named multiresolution neural network (MuReNN). The key idea behind MuReNN is to train separate convolutional operators over the octave subbands of a discrete wavelet transform (DWT). Since the scale of DWT atoms grows exponentially between octaves, the receptive fields of the subsequent learnable convolutions in MuReNN are dilated accordingly. For a given real-world dataset, we fit the magnitude response of MuReNN to that of a well-established auditory filterbank: Gammatone for speech, CQT for music, and third-octave for urban sounds, respectively. This is a form of knowledge distillation (KD), in which the filterbank ''teacher'' is engineered by domain knowledge while the neural network ''student'' is optimized from data. We compare MuReNN to the state of the art in terms of goodness of fit after KD on a hold-out set and in terms of Heisenberg time-frequency localization. Compared to convnets and Gabor convolutions, we find that MuReNN reaches state-of-the-art performance on all three optimization problems.