# Learnable Adaptive Time-Frequency Representation via Differentiable Short-Time Fourier Transform

arXiv: https://arxiv.org/abs/2506.21440 · HTML: https://mlsanity.com/paper/2506.21440/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 46 references that are in the map.
- First posted on arXiv 26 Jun 2025.

## Builds on (most cited first, 10 of 10)

- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [Hybrid Spectrogram and Waveform Source Separation](https://mlsanity.com/paper/2111.03600/) (2021)
- [Unsupervised speech enhancement with diffusion-based generative models](https://mlsanity.com/paper/2309.10450/) (2023)
- [Optimizing Short-Time Fourier Transform Parameters via Gradient Descent](https://mlsanity.com/paper/2010.15049/) (2020)
- [A differentiable short-time Fourier transform with respect to the window length](https://mlsanity.com/paper/2208.10886/) (2022)
- [Differentiable adaptive short-time Fourier transform with respect to the window length](https://mlsanity.com/paper/2308.02418/) (2023)
- [Adaptive Short-time Fourier Transform and Synchrosqueezing Transform for Non-stationary Signal Separation](https://mlsanity.com/paper/1812.11292/) (2018)
- [Adaptive Synchrosqueezing Transform with a Time-Varying Parameter for Non-stationary Signal Separation](https://mlsanity.com/paper/1812.11364/) (2018)
- [Differentiable short-time Fourier transform with respect to the hop length](https://mlsanity.com/paper/2308.02421/) (2023)

## Abstract (arXiv)

The short-time Fourier transform (STFT) is widely used for analyzing non-stationary signals. However, its performance is highly sensitive to its parameters, and manual or heuristic tuning often yields suboptimal results. To overcome this limitation, we propose a unified differentiable formulation of the STFT that enables gradient-based optimization of its parameters. This approach addresses the limitations of traditional STFT parameter tuning methods, which often rely on computationally intensive discrete searches. It enables fine-tuning of the time-frequency representation (TFR) based on any desired criterion. Moreover, our approach integrates seamlessly with neural networks, allowing joint optimization of the STFT parameters and network weights. The efficacy of the proposed differentiable STFT in enhancing TFRs and improving performance in downstream tasks is demonstrated through experiments on both simulated and real-world data.