# End-to-End Speech Recognition From the Raw Waveform

arXiv: https://arxiv.org/abs/1806.07098 · HTML: https://mlsanity.com/paper/1806.07098/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2018.

## Built on by (most cited first, 10 of 10)

- [End-to-end Audio-visual Speech Recognition with Conformers](https://mlsanity.com/paper/2102.06657/) (2021)
- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [LEAF: A Learnable Frontend for Audio Classification](https://mlsanity.com/paper/2101.08596/) (2021)
- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [RepAugment: Input-Agnostic Representation-Level Augmentation for Respiratory Sound Classification](https://mlsanity.com/paper/2405.02996/) (2024)
- [Interpreting intermediate convolutional layers of generative CNNs trained on waveforms](https://mlsanity.com/paper/2104.09489/) (2021)
- [Raw Waveform Encoder with Multi-Scale Globally Attentive Locally Recurrent Networks for End-to-End Speech Recognition](https://mlsanity.com/paper/2106.04275/) (2021)
- [Shennong: a Python toolbox for audio speech features extraction](https://mlsanity.com/paper/2112.05555/) (2021)
- [Learning neural audio features without supervision](https://mlsanity.com/paper/2203.15519/) (2022)
- [Learning Sparse Analytic Filters for Piano Transcription](https://mlsanity.com/paper/2108.10382/) (2021)

## Abstract (arXiv)

State-of-the-art speech recognition systems rely on fixed, hand-crafted features such as mel-filterbanks to preprocess the waveform before the training pipeline. In this paper, we study end-to-end systems trained directly from the raw waveform, building on two alternatives for trainable replacements of mel-filterbanks that use a convolutional architecture. The first one is inspired by gammatone filterbanks (Hoshen et al., 2015; Sainath et al, 2015), and the second one by the scattering transform (Zeghidour et al., 2017). We propose two modifications to these architectures and systematically compare them to mel-filterbanks, on the Wall Street Journal dataset. The first modification is the addition of an instance normalization layer, which greatly improves on the gammatone-based trainable filterbanks and speeds up the training of the scattering-based filterbanks. The second one relates to the low-pass filter used in these approaches. These modifications consistently improve performances for both approaches, and remove the need for a careful initialization in scattering-based trainable filterbanks. In particular, we show a consistent improvement in word error rate of the trainable filterbanks relatively to comparable mel-filterbanks. It is the first time end-to-end models trained from the raw signal significantly outperform mel-filterbanks on a large vocabulary task under clean recording conditions.