Score-Based Training for Energy-Based TTS Models
Not on the map. Why?
Abstract (arXiv)
Noise contrastive estimation (NCE) is a popular method for training energy-based models (EBM) with intractable normalisation terms. The key idea of NCE is to learn by comparing unnormalised log-likelihoods of the reference and noisy samples, thus avoiding explicitly computing normalisation terms. However, NCE critically relies on the quality of noisy samples. Recently, sliced score matching (SSM) has been popularised by closely related diffusion models (DM). Unlike NCE, SSM learns a gradient of log-likelihood, or score, by learning distribution of its projections on randomly chosen directions. However, both NCE and SSM disregard the form of log-likelihood function, which is problematic given that EBMs and DMs make use of first-order optimisation during inference. This paper proposes a new criterion that learns scores more suitable for first-order schemes. Experiments contrasts these approaches for training EBMs.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 33 references that are in the map. First posted on arXiv 19 May 2025.
Builds on
22 of 33 references.
References · 22
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
- 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
- 2021How to Train Your Energy-Based Modelscited by 274
- 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
10 more
- 2023Matcha-TTS: A fast TTS architecture with conditional flow matchingcited by 145
- 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 83
- 2023Iterative α-(de)Blending: a Minimalist Deterministic Diffusion Modelcited by 46
- 2020Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Modelscited by 44
- 2024SpeechBERTScore: Reference-Aware Automatic Evaluation of Speech Generation Leveraging NLP Evaluation Metricscited by 38
- 2018Learning deep kernels for exponential family densitiescited by 36
- 2021On Out-of-distribution Detection with Energy-based Modelscited by 17
- 2021Residual Energy-Based Models for End-to-End Speech Recognitioncited by 9
- 2020Imitation with Neural Density Modelscited by 8
- 2023Energy-Based Models For Speech Synthesiscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Learning normalized image densities via dual score matching
- 2025Heteroscedasticity of Denoising Score Matching with Generalised Smooth Noise
- 2021Heavy-tailed denoising score matching
- 2023Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood
- 2025Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High-Quality Generation
- 2026A Diffusive Classification Loss for Learning Energy-based Generative Models
- 2024Nonlinear denoising score matching for enhanced learning of structured distributions
- 2022Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matching