# Optical Diffusion Models for Image Generation

arXiv: https://arxiv.org/abs/2407.10897 · HTML: https://mlsanity.com/paper/2407.10897/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 25 of its 43 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 25 of 25)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Scaling Laws for Autoregressive Generative Modeling](https://mlsanity.com/paper/2010.14701/) (2020)
- [Robust Compressed Sensing MRI with Deep Generative Priors](https://mlsanity.com/paper/2108.01368/) (2021)
- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](https://mlsanity.com/paper/2311.16863/) (2023)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [On Fast Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2106.00132/) (2021)
- [Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems](https://mlsanity.com/paper/2104.13386/) (2021)
- [Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit](https://mlsanity.com/paper/2008.11659/) (2020)
- [The physics of optical computing](https://mlsanity.com/paper/2308.00088/) (2023)
- [Super-resolution image display using diffractive decoders](https://mlsanity.com/paper/2206.07281/) (2022)
- [Spatially Varying Nanophotonic Neural Networks](https://mlsanity.com/paper/2308.03407/) (2023)
- [Optical Transformers](https://mlsanity.com/paper/2302.10360/) (2023)
- [All-optical image denoising using a diffractive visual processor](https://mlsanity.com/paper/2309.09215/) (2023)

## Built on by (most cited first, 4 of 4)

- [Optical Generative Models](https://mlsanity.com/paper/2410.17970/) (2024)
- [Local Learning Rules for Out-of-Equilibrium Physical Generative Models](https://mlsanity.com/paper/2506.19136/) (2025)
- [Generative Models on Analog Hardware with Dynamics](https://mlsanity.com/paper/2606.27294/) (2026)
- [PotentialEP: Backprop-Free Training of CIFAR-Scale Generative Models via Equilibrium Propagation](https://mlsanity.com/paper/2610.14409/) (2026)

## Abstract (arXiv)

Diffusion models generate new samples by progressively decreasing the noise from the initially provided random distribution. This inference procedure generally utilizes a trained neural network numerous times to obtain the final output, creating significant latency and energy consumption on digital electronic hardware such as GPUs. In this study, we demonstrate that the propagation of a light beam through a semi-transparent medium can be programmed to implement a denoising diffusion model on image samples. This framework projects noisy image patterns through passive diffractive optical layers, which collectively only transmit the predicted noise term in the image. The optical transparent layers, which are trained with an online training approach, backpropagating the error to the analytical model of the system, are passive and kept the same across different steps of denoising. Hence this method enables high-speed image generation with minimal power consumption, benefiting from the bandwidth and energy efficiency of optical information processing.