# Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation

arXiv: https://arxiv.org/abs/2505.22391 · HTML: https://mlsanity.com/paper/2505.22391/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 52 of its 82 references that are in the map.
- First posted on arXiv 28 May 2025.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 30 of 52)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation](https://mlsanity.com/paper/2309.06380/) (2023)
- [Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed](https://mlsanity.com/paper/2101.02388/) (2021)
- [Denoising Diffusion Models for Plug-and-Play Image Restoration](https://mlsanity.com/paper/2305.08995/) (2023)
- [Fast Sampling of Diffusion Models via Operator Learning](https://mlsanity.com/paper/2211.13449/) (2022)
- [Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing](https://mlsanity.com/paper/2407.01521/) (2024)
- [A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction](https://mlsanity.com/paper/2211.14680/) (2022)
- [InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization](https://mlsanity.com/paper/2404.04650/) (2024)
- [D-Flow: Differentiating through Flows for Controlled Generation](https://mlsanity.com/paper/2402.14017/) (2024)
- [TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation](https://mlsanity.com/paper/2303.04248/) (2023)
- [Parallel Sampling of Diffusion Models](https://mlsanity.com/paper/2305.16317/) (2023)

## Built on by (most cited first, 11 of 11)

- [Physics vs Distributions: Pareto Optimal Flow Matching with Physics Constraints](https://mlsanity.com/paper/2506.08604/) (2025)
- [Physics-Constrained Fine-Tuning of Flow-Matching Models for Generation and Inverse Problems](https://mlsanity.com/paper/2508.09156/) (2025)
- [PRISMA: Improving the Accuracy-Latency Frontier of Diffusion-based PDE Solvers Using Physics-Informed Spectral Attention](https://mlsanity.com/paper/2512.01370/) (2025)
- [PILD: Physics-Informed Learning via Diffusion](https://mlsanity.com/paper/2601.21284/) (2026)
- [Ultra Fast PDE Solving via Physics Guided Few-step Diffusion](https://mlsanity.com/paper/2602.03627/) (2026)
- [Physics-Informed Neural PDE Solvers via Spatio-Temporal MeanFlow](https://mlsanity.com/paper/2605.08915/) (2026)
- [GenVoid: Uncertainty-Aware Learning of Subsurface Material Defects with an Experimentally Validated Physics-Informed Generative Model](https://mlsanity.com/paper/2609.23761/) (2026)
- [UniFluids: Unified Neural Operator Learning with Conditional Flow-matching](https://mlsanity.com/paper/2603.22309/) (2026)
- [Multiphase-Diff: Diffusion-Based Generative Modeling for High-Contrast Multiphase Physical Systems with Sharp Interfaces](https://mlsanity.com/paper/2608.13669/) (2026)
- [PMosFM: Preconditioned Manifold Matching for One-Step Physics-Constrained Generation](https://mlsanity.com/paper/2609.40287/) (2026)
- [Transferability of Learned States in Neural PDE Solvers](https://mlsanity.com/paper/2610.10972/) (2026)

## Abstract (arXiv)

Modeling physical systems in a generative manner offers several advantages, including the ability to handle partial observations, generate diverse solutions, and address both forward and inverse problems. Recently, diffusion models have gained increasing attention in the modeling of physical systems, particularly those governed by partial differential equations (PDEs). However, diffusion models only access noisy data boldsymbolxt at intermediate steps, making it infeasible to directly enforce constraints on the clean sample boldsymbolx0 at each noisy level. As a workaround, constraints are typically applied to the expectation of clean samples mathbbE[boldsymbolx0|boldsymbolxt], which is estimated using the learned score network. However, imposing PDE constraints on the expectation does not strictly represent the one on the true clean data, known as Jensen's Gap. This gap creates a trade-off: enforcing PDE constraints may come at the cost of reduced accuracy in generative modeling. To address this, we propose a simple yet effective post-hoc distillation approach, where PDE constraints are not injected directly into the diffusion process, but instead enforced during a post-hoc distillation stage. We term our method as Physics-Informed Distillation of Diffusion Models (PIDDM). This distillation not only facilitates single-step generation with improved PDE satisfaction, but also support both forward and inverse problem solving and reconstruction from randomly partial observation. Extensive experiments across various PDE benchmarks demonstrate that PIDDM significantly improves PDE satisfaction over several recent and competitive baselines, such as PIDM, DiffusionPDE, and ECI-sampling, with less computation overhead. Our approach can shed light on more efficient and effective strategies for incorporating physical constraints into diffusion models.