# Biomimetic Frontend for Differentiable Audio Processing

arXiv: https://arxiv.org/abs/2409.08997 · HTML: https://mlsanity.com/paper/2409.08997/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 48 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 14 of 14)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [LEAF: A Learnable Frontend for Audio Classification](https://mlsanity.com/paper/2101.08596/) (2021)
- [Physics-based Deep Learning](https://mlsanity.com/paper/2109.05237/) (2021)
- [Diffusion-based Generative Speech Source Separation](https://mlsanity.com/paper/2210.17327/) (2022)
- [Trainable Frontend For Robust and Far-Field Keyword Spotting](https://mlsanity.com/paper/1607.05666/) (2016)
- [Characterizing the adversarial vulnerability of speech self-supervised learning](https://mlsanity.com/paper/2111.04330/) (2021)
- [A Modulation-Domain Loss for Neural-Network-based Real-time Speech Enhancement](https://mlsanity.com/paper/2102.07330/) (2021)
- [A Neural-Network Framework for the Design of Individualised Hearing-Loss Compensation](https://mlsanity.com/paper/2207.07091/) (2022)
- [What is Learnt by the LEArnable Front-end (LEAF)? Adapting Per-Channel Energy Normalisation (PCEN) to Noisy Conditions](https://mlsanity.com/paper/2404.06702/) (2024)
- [Learnable Spectro-temporal Receptive Fields for Robust Voice Type Discrimination](https://mlsanity.com/paper/2010.09151/) (2020)

## Abstract (arXiv)

While models in audio and speech processing are becoming deeper and more end-to-end, they as a consequence need expensive training on large data, and are often brittle. We build on a classical model of human hearing and make it differentiable, so that we can combine traditional explainable biomimetic signal processing approaches with deep-learning frameworks. This allows us to arrive at an expressive and explainable model that is easily trained on modest amounts of data. We apply this model to audio processing tasks, including classification and enhancement. Results show that our differentiable model surpasses black-box approaches in terms of computational efficiency and robustness, even with little training data. We also discuss other potential applications.