# Diffusion-Model-Assisted Supervised Learning of Generative Models for Density Estimation

arXiv: https://arxiv.org/abs/2310.14458 · HTML: https://mlsanity.com/paper/2310.14458/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 28 of its 36 references that are in the map.
- First posted on arXiv 22 Oct 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019)
- [Learning Gradient Fields for Shape Generation](https://mlsanity.com/paper/2008.06520/) (2020)
- [Step-unrolled Denoising Autoencoders for Text Generation](https://mlsanity.com/paper/2112.06749/) (2021)
- [FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flow](https://mlsanity.com/paper/1909.02480/) (2019)
- [Score-Based Point Cloud Denoising](https://mlsanity.com/paper/2107.10981/) (2021)
- [Latent Diffusion Energy-Based Model for Interpretable Text Modeling](https://mlsanity.com/paper/2206.05895/) (2022)
- [Conditional Simulation Using Diffusion Schr"odinger Bridges](https://mlsanity.com/paper/2202.13460/) (2022)
- [Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models](https://mlsanity.com/paper/2108.12956/) (2021)
- [A Score-based Nonlinear Filter for Data Assimilation](https://mlsanity.com/paper/2306.09282/) (2023)

## Built on by (most cited first, 13 of 13)

- [A Training-Free Conditional Diffusion Model for Learning Stochastic Dynamical Systems](https://mlsanity.com/paper/2410.03108/) (2024)
- [Bifidelity Parameter Estimation Using Conditional Diffusion Models](https://mlsanity.com/paper/2504.01894/) (2025)
- [A Scalable Real-Time Data Assimilation Framework for Predicting Turbulent Atmosphere Dynamics](https://mlsanity.com/paper/2407.12168/) (2024)
- [Generative Learning of Densities on Manifolds](https://mlsanity.com/paper/2503.03963/) (2025)
- [Generative Learning for Slow Manifolds and Bifurcation Diagrams](https://mlsanity.com/paper/2504.20375/) (2025)
- [Generative AI Models for Learning Flow Maps of Stochastic Dynamical Systems in Bounded Domains](https://mlsanity.com/paper/2507.15990/) (2025)
- [A Score Filter Enhanced Data Assimilation Framework for Data-Driven Dynamical Systems](https://mlsanity.com/paper/2603.14863/) (2026)
- [Diffusion-based supervised learning of generative models for efficient sampling of multimodal distributions](https://mlsanity.com/paper/2505.07825/) (2025)
- [Guided Unconditional and Conditional Generative Models for Super-Resolution and Inference of Quasi-Geostrophic Turbulence](https://mlsanity.com/paper/2507.00719/) (2025)
- [Conformal Disentanglement and Latent-Space Curation: A Neural Framework for Perspective Synthesis, Differentiation and Targeted Generation](https://mlsanity.com/paper/2408.15344/) (2024)
- [Learning Stochastic Hamiltonian Systems via Stochastic Generating Function Neural Network](https://mlsanity.com/paper/2507.14467/) (2025)
- [Training-free score-based diffusion for parameter-dependent stochastic dynamical systems](https://mlsanity.com/paper/2602.02113/) (2026)
- [Bringing together invertible UNets with invertible attention modules for memory-efficient diffusion models](https://mlsanity.com/paper/2504.10883/) (2025)

## Abstract (arXiv)

We present a supervised learning framework of training generative models for density estimation. Generative models, including generative adversarial networks, normalizing flows, variational auto-encoders, are usually considered as unsupervised learning models, because labeled data are usually unavailable for training. Despite the success of the generative models, there are several issues with the unsupervised training, e.g., requirement of reversible architectures, vanishing gradients, and training instability. To enable supervised learning in generative models, we utilize the score-based diffusion model to generate labeled data. Unlike existing diffusion models that train neural networks to learn the score function, we develop a training-free score estimation method. This approach uses mini-batch-based Monte Carlo estimators to directly approximate the score function at any spatial-temporal location in solving an ordinary differential equation (ODE), corresponding to the reverse-time stochastic differential equation (SDE). This approach can offer both high accuracy and substantial time savings in neural network training. Once the labeled data are generated, we can train a simple fully connected neural network to learn the generative model in the supervised manner. Compared with existing normalizing flow models, our method does not require to use reversible neural networks and avoids the computation of the Jacobian matrix. Compared with existing diffusion models, our method does not need to solve the reverse-time SDE to generate new samples. As a result, the sampling efficiency is significantly improved. We demonstrate the performance of our method by applying it to a set of 2D datasets as well as real data from the UCI repository.