# Learning Multiscale Features Directly From Waveforms

arXiv: https://arxiv.org/abs/1603.09509 · HTML: https://mlsanity.com/paper/1603.09509/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2016.

## Built on by (most cited first, 9 of 9)

- [On the limit of English conversational speech recognition](https://mlsanity.com/paper/2105.00982/) (2021)
- [Towards Signal Processing In Large Language Models](https://mlsanity.com/paper/2406.10254/) (2024)
- [MAST: Multiscale Audio Spectrogram Transformers](https://mlsanity.com/paper/2211.01515/) (2022)
- [Raw Waveform Encoder with Multi-Scale Globally Attentive Locally Recurrent Networks for End-to-End Speech Recognition](https://mlsanity.com/paper/2106.04275/) (2021)
- [MuSLCAT: Multi-Scale Multi-Level Convolutional Attention Transformer for Discriminative Music Modeling on Raw Waveforms](https://mlsanity.com/paper/2104.02309/) (2021)
- [A Modulation Front-End for Music Audio Tagging](https://mlsanity.com/paper/2105.11836/) (2021)
- [Improving Primate Sounds Classification using Binary Presorting for Deep Learning](https://mlsanity.com/paper/2306.16054/) (2023)
- [Phonetic Error Analysis of Raw Waveform Acoustic Models with Parametric and Non-Parametric CNNs](https://mlsanity.com/paper/2406.00898/) (2024)
- [Phonetic Error Analysis of Raw Waveform Acoustic Models](https://mlsanity.com/paper/2606.07030/) (2026)

## Abstract (arXiv)

Deep learning has dramatically improved the performance of speech recognition systems through learning hierarchies of features optimized for the task at hand. However, true end-to-end learning, where features are learned directly from waveforms, has only recently reached the performance of hand-tailored representations based on the Fourier transform. In this paper, we detail an approach to use convolutional filters to push past the inherent tradeoff of temporal and frequency resolution that exists for spectral representations. At increased computational cost, we show that increasing temporal resolution via reduced stride and increasing frequency resolution via additional filters delivers significant performance improvements. Further, we find more efficient representations by simultaneously learning at multiple scales, leading to an overall decrease in word error rate on a difficult internal speech test set by 20.7% relative to networks with the same number of parameters trained on spectrograms.