# STFT spectral loss for training a neural speech waveform model

arXiv: https://arxiv.org/abs/1810.11945 · HTML: https://mlsanity.com/paper/1810.11945/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2018.

## Built on by (most cited first, 4 of 4)

- [Cross Attention Transformers for Multi-modal Unsupervised Whole-Body PET Anomaly Detection](https://mlsanity.com/paper/2304.07147/) (2023)
- [SaMoye: Zero-shot Singing Voice Conversion Model Based on Feature Disentanglement and Enhancement](https://mlsanity.com/paper/2407.07728/) (2024)
- [WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modeling](https://mlsanity.com/paper/2606.03455/) (2026)
- [SIMD-size aware weight regularization for fast neural vocoding on CPU](https://mlsanity.com/paper/2211.00898/) (2022)

## Abstract (arXiv)

This paper proposes a new loss using short-time Fourier transform (STFT) spectra for the aim of training a high-performance neural speech waveform model that predicts raw continuous speech waveform samples directly. Not only amplitude spectra but also phase spectra obtained from generated speech waveforms are used to calculate the proposed loss. We also mathematically show that training of the waveform model on the basis of the proposed loss can be interpreted as maximum likelihood training that assumes the amplitude and phase spectra of generated speech waveforms following Gaussian and von Mises distributions, respectively. Furthermore, this paper presents a simple network architecture as the speech waveform model, which is composed of uni-directional long short-term memories (LSTMs) and an auto-regressive structure. Experimental results showed that the proposed neural model synthesized high-quality speech waveforms.