# Learning Filterbanks from Raw Speech for Phone Recognition

arXiv: https://arxiv.org/abs/1711.01161 · HTML: https://mlsanity.com/paper/1711.01161/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2017.

## Built on by (most cited first, 25 of 25)

- [LEAF: A Learnable Frontend for Audio Classification](https://mlsanity.com/paper/2101.08596/) (2021)
- [How to Design a Three-Stage Architecture for Audio-Visual Active Speaker Detection in the Wild](https://mlsanity.com/paper/2106.03932/) (2021)
- [Known Operator Learning and Hybrid Machine Learning in Medical Imaging -- A Review of the Past, the Present, and the Future](https://mlsanity.com/paper/2108.04543/) (2021)
- [I Can Hear You: Selective Robust Training for Deepfake Audio Detection](https://mlsanity.com/paper/2411.00121/) (2024)
- [FastAudio: A Learnable Audio Front-End for Spoof Speech Detection](https://mlsanity.com/paper/2109.02774/) (2021)
- [EfficientLEAF: A Faster LEarnable Audio Frontend of Questionable Use](https://mlsanity.com/paper/2207.05508/) (2022)
- [Learnable MFCCs for Speaker Verification](https://mlsanity.com/paper/2102.10322/) (2021)
- [Learning Temporal Resolution in Spectrogram for Audio Classification](https://mlsanity.com/paper/2210.01719/) (2022)
- [Fitting Auditory Filterbanks with Multiresolution Neural Networks](https://mlsanity.com/paper/2307.13821/) (2023)
- [Raw Waveform Encoder with Multi-Scale Globally Attentive Locally Recurrent Networks for End-to-End Speech Recognition](https://mlsanity.com/paper/2106.04275/) (2021)
- [Instabilities in Convnets for Raw Audio](https://mlsanity.com/paper/2309.05855/) (2023)
- [TACNET: Temporal Audio Source Counting Network](https://mlsanity.com/paper/2311.02369/) (2023)
- [Regularizing Learnable Feature Extraction for Automatic Speech Recognition](https://mlsanity.com/paper/2506.09804/) (2025)
- [Learnable Nonlinear Compression for Robust Speaker Verification](https://mlsanity.com/paper/2202.05236/) (2022)
- [Learning neural audio features without supervision](https://mlsanity.com/paper/2203.15519/) (2022)
- [A Modulation Front-End for Music Audio Tagging](https://mlsanity.com/paper/2105.11836/) (2021)
- [Learning Sparse Analytic Filters for Piano Transcription](https://mlsanity.com/paper/2108.10382/) (2021)
- [Low-Level Physiological Implications of End-to-End Learning of Speech Recognition](https://mlsanity.com/paper/2208.11700/) (2022)
- [Towards Enhanced Classification of Abnormal Lung sound in Multi-breath: A Light Weight Multi-label and Multi-head Attention Classification Method](https://mlsanity.com/paper/2407.10828/) (2024)
- [A Joint Spectro-Temporal Relational Thinking Based Acoustic Modeling Framework](https://mlsanity.com/paper/2409.15357/) (2024)
- [Raw Audio Classification with Cosine Convolutional Neural Network (CosCovNN)](https://mlsanity.com/paper/2412.00312/) (2024)
- [ISAC: An Invertible and Stable Auditory Filter Bank with Customizable Kernels for ML Integration](https://mlsanity.com/paper/2505.07709/) (2025)
- [LearnAFE: Circuit-Algorithm Co-design Framework for Learnable Audio Analog Front-End](https://mlsanity.com/paper/2507.00755/) (2025)
- [AUDETER: A Large-scale Dataset for Deepfake Audio Detection in Open Worlds](https://mlsanity.com/paper/2509.04345/) (2025)
- [Unified Learnable 2D Convolutional Feature Extraction for ASR](https://mlsanity.com/paper/2509.10031/) (2025)

## Abstract (arXiv)

We train a bank of complex filters that operates on the raw waveform and is fed into a convolutional neural network for end-to-end phone recognition. These time-domain filterbanks (TD-filterbanks) are initialized as an approximation of mel-filterbanks, and then fine-tuned jointly with the remaining convolutional architecture. We perform phone recognition experiments on TIMIT and show that for several architectures, models trained on TD-filterbanks consistently outperform their counterparts trained on comparable mel-filterbanks. We get our best performance by learning all front-end steps, from pre-emphasis up to averaging. Finally, we observe that the filters at convergence have an asymmetric impulse response, and that some of them remain almost analytic.