# Efficient Neural and Numerical Methods for High-Quality Online Speech Spectrogram Inversion via Gradient Theorem

arXiv: https://arxiv.org/abs/2505.24498 · HTML: https://mlsanity.com/paper/2505.24498/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 47 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 20 of 20)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Deep Speech 2: End-to-End Speech Recognition in English and Mandarin](https://mlsanity.com/paper/1512.02595/) (2015)
- [Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis](https://mlsanity.com/paper/2306.00814/) (2023)
- [GANSynth: Adversarial Neural Audio Synthesis](https://mlsanity.com/paper/1902.08710/) (2019)
- [T-GSA: Transformer with Gaussian-weighted self-attention for speech enhancement](https://mlsanity.com/paper/1910.06762/) (2019)
- [Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks](https://mlsanity.com/paper/1808.06719/) (2018)
- [SubSpectral Normalization for Neural Audio Data Processing](https://mlsanity.com/paper/2103.13620/) (2021)
- [End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction](https://mlsanity.com/paper/1804.10204/) (2018)
- [A Non-iterative Method for (Re)Construction of Phase from STFT Magnitude](https://mlsanity.com/paper/1609.00291/) (2016)
- [Phase reconstruction from amplitude spectrograms based on von-Mises-distribution deep neural network](https://mlsanity.com/paper/1807.03474/) (2018)
- [Phasebook and Friends: Leveraging Discrete Representations for Source Separation](https://mlsanity.com/paper/1810.01395/) (2018)
- [A Flexible Online Framework for Projection-Based STFT Phase Retrieval](https://mlsanity.com/paper/2309.07043/) (2023)
- [Phase retrieval with Bregman divergences and application to audio signal recovery](https://mlsanity.com/paper/2010.00392/) (2020)
- [Online Phase Reconstruction via DNN-based Phase Differences Estimation](https://mlsanity.com/paper/2211.08246/) (2022)

## Abstract (arXiv)

Recent work in online speech spectrogram inversion effectively combines Deep Learning with the Gradient Theorem to predict phase derivatives directly from magnitudes. Then, phases are estimated from their derivatives via least squares, resulting in a high quality reconstruction. In this work, we introduce three innovations that drastically reduce computational cost, while maintaining high quality: Firstly, we introduce a novel neural network architecture with just 8k parameters, 30 times smaller than previous state of the art. Secondly, increasing latency by 1 hop size allows us to further halve the cost of the neural inference step. Thirdly, we we observe that the least squares problem features a tridiagonal matrix and propose a linear-complexity solver for the least squares step that leverages tridiagonality and positive-semidefiniteness, achieving a speedup of several orders of magnitude. We release samples online.