# Regularizing Learnable Feature Extraction for Automatic Speech Recognition

arXiv: https://arxiv.org/abs/2506.09804 · HTML: https://mlsanity.com/paper/2506.09804/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 11 Jun 2025.

## Builds on (most cited first, 17 of 17)

- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [LEAF: A Learnable Frontend for Audio Classification](https://mlsanity.com/paper/2101.08596/) (2021)
- [RETURNN as a Generic Flexible Neural Toolkit with Application to Translation and Speech Recognition](https://mlsanity.com/paper/1805.05225/) (2018)
- [Learning Filterbanks from Raw Speech for Phone Recognition](https://mlsanity.com/paper/1711.01161/) (2017)
- [Semantic Mask for Transformer based End-to-End Speech Recognition](https://mlsanity.com/paper/1912.03010/) (2019)
- [Efficient Training of Neural Transducer for Speech Recognition](https://mlsanity.com/paper/2204.10586/) (2022)
- [SpecAugment++: A Hidden Space Data Augmentation Method for Acoustic Scene Classification](https://mlsanity.com/paper/2103.16858/) (2021)
- [Data Augmentation For Children's Speech Recognition -- The "Ethiopian" System For The SLT 2021 Children Speech Recognition Challenge](https://mlsanity.com/paper/2011.04547/) (2020)
- [On Architectures and Training for Raw Waveform Feature Extraction in ASR](https://mlsanity.com/paper/2104.04298/) (2021)
- [SpliceOut: A Simple and Efficient Audio Augmentation Method](https://mlsanity.com/paper/2110.00046/) (2021)
- [Data Augmentation with Signal Companding for Detection of Logical Access Attacks](https://mlsanity.com/paper/2102.06332/) (2021)
- [Comparative Analysis of the wav2vec 2.0 Feature Extractor](https://mlsanity.com/paper/2308.04286/) (2023)
- [RASR2: The RWTH ASR Toolkit for Generic Sequence-to-sequence Speech Recognition](https://mlsanity.com/paper/2305.17782/) (2023)
- [Improving Speech Representation Learning via Speech-level and Phoneme-level Masking Approach](https://mlsanity.com/paper/2210.13805/) (2022)

## Built on by (most cited first, 2 of 2)

- [Unified Learnable 2D Convolutional Feature Extraction for ASR](https://mlsanity.com/paper/2509.10031/) (2025)
- [Error Rate is Not Enough: Costs of Augmentation in Low-Resource ASR](https://mlsanity.com/paper/2610.13807/) (2026)

## Abstract (arXiv)

Neural front-ends are an appealing alternative to traditional, fixed feature extraction pipelines for automatic speech recognition (ASR) systems since they can be directly trained to fit the acoustic model. However, their performance often falls short compared to classical methods, which we show is largely due to their increased susceptibility to overfitting. This work therefore investigates regularization methods for training ASR models with learnable feature extraction front-ends. First, we examine audio perturbation methods and show that larger relative improvements can be obtained for learnable features. Additionally, we identify two limitations in the standard use of SpecAugment for these front-ends and propose masking in the short time Fourier transform (STFT)-domain as a simple but effective modification to address these challenges. Finally, integrating both regularization approaches effectively closes the performance gap between traditional and learnable features.