mlsanityData to 7 Oct 2026

Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inference

arXiv 2407.17033 ↗First posted 24 Jul 2024cs.LG cs.AI stat.MLcited by 16

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Deep Gaussian processes (DGPs) provide a robust paradigm for Bayesian deep learning. In DGPs, a set of sparse integration locations called inducing points are selected to approximate the posterior distribution of the model. This is done to reduce computational complexity and improve model efficiency. However, inferring the posterior distribution of inducing points is not straightforward. Traditional variational inference approaches to posterior approximation often lead to significant bias. To address this issue, we propose an alternative method called Denoising Diffusion Variational Inference (DDVI) that uses a denoising diffusion stochastic differential equation (SDE) to generate posterior samples of inducing variables. We rely on score matching methods for denoising diffusion model to approximate score functions with a neural network. Furthermore, by combining classical mathematical theory of SDEs with the minimization of KL divergence between the approximate and true processes, we propose a novel explicit variational lower bound for the marginal likelihood function of DGP. Through experiments on various datasets and comparisons with baseline methods, we empirically demonstrate the effectiveness of DDVI for posterior inference of inducing points for DGP models.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 35 of its 57 references that are in the map. First posted on arXiv 24 Jul 2024.

Builds on

35 of 57 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Denoising Diffusion Samplers27 Feb
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  4. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2016Variational Inference: A Review for Statisticianscited by 1,466
  6. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  7. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 341
  8. 2013A survey of the Schr"odinger problem and some of its connections with optimal transportcited by 286
  9. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  10. 2017Advances in Variational Inferencecited by 236
  11. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
  12. 2017Doubly Stochastic Variational Inference for Deep Gaussian Processescited by 143
19 more
  1. 2019Functional Variational Bayesian Neural Networkscited by 127
  2. 2017Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networkscited by 107
  3. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  4. 2019Theoretical guarantees for sampling and inference in generative models with latent diffusionscited by 97
  5. 2016Stochastic Variational Deep Kernel Learningcited by 92
  6. 2021Efficient and Accurate Gradients for Neural SDEscited by 68
  7. 2020Markovian Score Climbing: Variational Inference with KL(p||q)cited by 46
  8. 2018Variational Implicit Processescited by 41
  9. 2018Inference in Deep Gaussian Processes using Stochastic Gradient Hamiltonian Monte Carlocited by 32
  10. 2019Approximate Inference for Fully Bayesian Gaussian Process Regressioncited by 31
  11. 2019On Stabilizing Generative Adversarial Training with Noisecited by 25
  12. 2014Structured Stochastic Variational Inferencecited by 21
  13. 2017Variational Inference for Gaussian Process Models with Linear Complexitycited by 21
  14. 2014Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Modelscited by 19
  15. 2018Orthogonally Decoupled Variational Gaussian Processescited by 19
  16. 2019Implicit Posterior Variational Inference for Deep Gaussian Processescited by 17
  17. 2021Structured Stochastic Gradient MCMCcited by 12
  18. 2021Scalable Variational Gaussian Processes via Harmonic Kernel Decompositioncited by 8
  19. 2022Deep Variational Implicit Processescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 7.

16 papers.

Sort:

By area:

  1. 2025Dequantified Diffusion-Schr"odinger Bridge for Density Ratio Estimationcited by 11
  2. 2025EVODiff: Entropy-aware Variance Optimized Diffusion Inferencecited by 10
  3. 2024Fully Bayesian Differential Gaussian Processes through Stochastic Differential Equationscited by 4
  4. 2025Diffusion Bridge Variational Inference for Deep Gaussian Processescited by 4
  5. 2024Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Samplingcited by 3
  6. 2026A Minimum Variance Path Principle for Accurate and Stable Score-Based Density Ratio Estimationcited by 3
  7. 2024Sparse Variational Student-t Processes for Heavy-tailed Modelingcited by 2
  8. 2026Mitigating the Contractivity Trap in Diffusion ODEs via Stein Stabilizationcited by 2
  9. 2026Towards Disentangled Preference Optimization Dynamics: Suppress the Loser, Preserve the Winnercited by 1
  10. 2026Onsager-Machlup Posterior Transport for Deep Gaussian Processescited by 1
  11. 2026DiFA: Inference-Time Forward-Process Alignment for Diffusion Modelscited by 1
  12. 2025Gaussian Process Tilted Nonparametric Density Estimation using Fisher Divergence Score Matchingcited by 0
4 more
  1. 2025Sparse Techniques for Regression in Deep Gaussian Processescited by 0
  2. 2025Neural Bridge Processescited by 0
  3. 2026Rethinking the Flow-Based Gradual Domain Adaptation: A Semi-Dual Optimal Transport Perspectivecited by 0
  4. 2026Implicit Variational Rejection Samplingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors
  2. 2023Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Models
  3. 2023To smooth a cloud or to pin it down: Guarantees and Insights on Score Matching in Denoising Diffusion Models
  4. 2025Low-dimensional adaptation of diffusion models: Convergence in total variation
  5. 2024Denoising Diffusion Probabilistic Models in Six Simple Steps
  6. 2024O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions
  7. 2025Diffusion Models: A Mathematical Introduction
  8. 2020Denoising Diffusion Implicit Models