# Neural Diffusion Processes

arXiv: https://arxiv.org/abs/2206.03992 · HTML: https://mlsanity.com/paper/2206.03992/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 8 Jun 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation](https://mlsanity.com/paper/2203.02923/) (2022)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [GP-VAE: Deep Probabilistic Time Series Imputation](https://mlsanity.com/paper/1907.04155/) (2019)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning](https://mlsanity.com/paper/2106.02584/) (2021)
- [Meta-Learning Stationary Stochastic Process Prediction with Convolutional Neural Processes](https://mlsanity.com/paper/2007.01332/) (2020)
- [A Framework for Interdomain and Multioutput Gaussian Processes](https://mlsanity.com/paper/2003.01115/) (2020)
- [The Gaussian Neural Process](https://mlsanity.com/paper/2101.03606/) (2021)
- [Approximate Inference for Fully Bayesian Gaussian Process Regression](https://mlsanity.com/paper/1912.13440/) (2019)
- [Kernel Identification Through Transformers](https://mlsanity.com/paper/2106.08185/) (2021)
- [Marginalised Gaussian Processes with Nested Sampling](https://mlsanity.com/paper/2010.16344/) (2020)

## Built on by (most cited first, 30 of 50)

- [A Survey on Generative Diffusion Model](https://mlsanity.com/paper/2209.02646/) (2022)
- [Score-based Diffusion Models in Function Space](https://mlsanity.com/paper/2302.07400/) (2023)
- [Diffusion Models for Black-Box Optimization](https://mlsanity.com/paper/2306.07180/) (2023)
- [Diffusion Generative Models in Infinite Dimensions](https://mlsanity.com/paper/2212.00886/) (2022)
- [DEFT: Efficient Fine-Tuning of Diffusion Models by Learning the Generalised h-transform](https://mlsanity.com/paper/2406.01781/) (2024)
- [Modeling Temporal Data as Continuous Functions with Stochastic Process Diffusion](https://mlsanity.com/paper/2211.02590/) (2022)
- [Continuous-Time Functional Diffusion Processes](https://mlsanity.com/paper/2303.00800/) (2023)
- [Conditional score-based diffusion models for Bayesian inference in infinite dimensions](https://mlsanity.com/paper/2305.19147/) (2023)
- [Spectral Diffusion Processes](https://mlsanity.com/paper/2209.14125/) (2022)
- [Amortized Probabilistic Conditioning for Optimization, Simulation and Inference](https://mlsanity.com/paper/2410.15320/) (2024)
- [A framework for conditional diffusion modelling with applications in motif scaffolding for protein design](https://mlsanity.com/paper/2312.09236/) (2023)
- [∞-Diff: Infinite Resolution Diffusion with Subsampled Mollified States](https://mlsanity.com/paper/2303.18242/) (2023)
- [FunDiff: Diffusion Models over Function Spaces for Physics-Informed Generative Modeling](https://mlsanity.com/paper/2506.07902/) (2025)
- [Stochastic Optimal Control for Diffusion Bridges in Function Spaces](https://mlsanity.com/paper/2405.20630/) (2024)
- [Stochastic Process Learning via Operator Flow Matching](https://mlsanity.com/paper/2501.04126/) (2025)
- [Functional Diffusion](https://mlsanity.com/paper/2311.15435/) (2023)
- [Manifold Diffusion Fields](https://mlsanity.com/paper/2305.15586/) (2023)
- [Universal Functional Regression with Neural Operator Flows](https://mlsanity.com/paper/2404.02986/) (2024)
- [Source Separation by Flow Matching](https://mlsanity.com/paper/2505.16119/) (2025)
- [Thompson Sampling with Diffusion Generative Prior](https://mlsanity.com/paper/2301.05182/) (2023)
- [Deep Stochastic Processes via Functional Markov Transition Operators](https://mlsanity.com/paper/2305.15574/) (2023)
- [Generative Hyperelasticity with Physics-Informed Probabilistic Diffusion Fields](https://mlsanity.com/paper/2310.03745/) (2023)
- [PriorCVAE: scalable MCMC parameter inference with Bayesian deep generative modelling](https://mlsanity.com/paper/2304.04307/) (2023)
- [Flow Straight and Fast in Hilbert Space: Functional Rectified Flow](https://mlsanity.com/paper/2509.10384/) (2025)
- [In-Context Multi-Objective Optimization](https://mlsanity.com/paper/2512.11114/) (2025)
- [Flow Matching Neural Processes](https://mlsanity.com/paper/2512.23853/) (2025)
- [The Alpha-Alternator: Dynamic Adaptation To Varying Noise Levels In Sequences Using The Vendi Score For Improved Robustness and Performance](https://mlsanity.com/paper/2502.04593/) (2025)
- [Dimension Agnostic Neural Processes](https://mlsanity.com/paper/2502.20661/) (2025)
- [Learning multi-modal generative models with permutation-invariant encoders and tighter variational objectives](https://mlsanity.com/paper/2309.00380/) (2023)
- [Spectral Convolutional Conditional Neural Processes](https://mlsanity.com/paper/2404.13182/) (2024)

## Abstract (arXiv)

Neural network approaches for meta-learning distributions over functions have desirable properties such as increased flexibility and a reduced complexity of inference. Building on the successes of denoising diffusion models for generative modelling, we propose Neural Diffusion Processes (NDPs), a novel approach that learns to sample from a rich distribution over functions through its finite marginals. By introducing a custom attention block we are able to incorporate properties of stochastic processes, such as exchangeability, directly into the NDP's architecture. We empirically show that NDPs can capture functional distributions close to the true Bayesian posterior, demonstrating that they can successfully emulate the behaviour of Gaussian processes and surpass the performance of neural processes. NDPs enable a variety of downstream tasks, including regression, implicit hyperparameter marginalisation, non-Gaussian posterior prediction and global optimisation.