# Gradient-Free Generation for Hard-Constrained Systems

arXiv: https://arxiv.org/abs/2412.01786 · HTML: https://mlsanity.com/paper/2412.01786/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 43 of its 49 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 30 of 43)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Discrete Flow Matching](https://mlsanity.com/paper/2407.15595/) (2024)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers](https://mlsanity.com/paper/2308.05732/) (2023)
- [Poseidon: Efficient Foundation Models for PDEs](https://mlsanity.com/paper/2405.19101/) (2024)
- [A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction](https://mlsanity.com/paper/2211.14680/) (2022)
- [Understanding and mitigating gradient pathologies in physics-informed neural networks](https://mlsanity.com/paper/2001.04536/) (2020)
- [D-Flow: Differentiating through Flows for Controlled Generation](https://mlsanity.com/paper/2402.14017/) (2024)
- [DiffusionPDE: Generative PDE-Solving Under Partial Observation](https://mlsanity.com/paper/2406.17763/) (2024)
- [Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2307.11494/) (2023)

## Built on by (most cited first, 30 of 38)

- [Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraints](https://mlsanity.com/paper/2506.04171/) (2025)
- [Physics vs Distributions: Pareto Optimal Flow Matching with Physics Constraints](https://mlsanity.com/paper/2506.08604/) (2025)
- [Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation](https://mlsanity.com/paper/2505.22391/) (2025)
- [Mesh-Informed Neural Operator : A Transformer Generative Approach](https://mlsanity.com/paper/2506.16656/) (2025)
- [Physics-Constrained Fine-Tuning of Flow-Matching Models for Generation and Inverse Problems](https://mlsanity.com/paper/2508.09156/) (2025)
- [Decoupled Diffusion Sampling for Inverse Problems on Function Spaces](https://mlsanity.com/paper/2601.23280/) (2026)
- [End-to-End Probabilistic Framework for Learning with Hard Constraints](https://mlsanity.com/paper/2506.07003/) (2025)
- [FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systems](https://mlsanity.com/paper/2505.17351/) (2025)
- [VideoPDE: Unified Generative PDE Solving via Video Inpainting Diffusion Models](https://mlsanity.com/paper/2506.13754/) (2025)
- [Chance-constrained Flow Matching for High-Fidelity Constraint-aware Generation](https://mlsanity.com/paper/2509.25157/) (2025)
- [Project and Generate: Divergence-Free Neural Operators for Incompressible Flows](https://mlsanity.com/paper/2603.24500/) (2026)
- [Strictly Constrained Generative Modeling via Split Augmented Langevin Sampling](https://mlsanity.com/paper/2505.18017/) (2025)
- [FD-Bench: A Modular and Fair Benchmark for Data-driven Fluid Simulation](https://mlsanity.com/paper/2505.20349/) (2025)
- [PRISMA: Improving the Accuracy-Latency Frontier of Diffusion-based PDE Solvers Using Physics-Informed Spectral Attention](https://mlsanity.com/paper/2512.01370/) (2025)
- [Flow-based Extremal Mathematical Structure Discovery](https://mlsanity.com/paper/2601.18005/) (2026)
- [Constraint-Aware Flow Matching: Decision Aligned End-to-End Training for Constrained Sampling](https://mlsanity.com/paper/2605.12754/) (2026)
- [PIRF: Physics-Informed Reward Fine-Tuning for Diffusion Models](https://mlsanity.com/paper/2509.20570/) (2025)
- [ProFlow: Zero-Shot Physics-Consistent Sampling via Proximal Flow Guidance](https://mlsanity.com/paper/2601.20227/) (2026)
- [Improved Constrained Generation by Bridging Pretrained Generative Models](https://mlsanity.com/paper/2603.06742/) (2026)
- [Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations](https://mlsanity.com/paper/2605.13790/) (2026)
- [Softly Constrained Denoisers for Diffusion Models Applied to Partial Differential Equations](https://mlsanity.com/paper/2512.14980/) (2025)
- [The Theory and Practice of MAP Inference over Non-Convex Constraints](https://mlsanity.com/paper/2602.08681/) (2026)
- [Constrained Diffusion for Accelerated Structure Relaxation of Inorganic Solids with Point Defects](https://mlsanity.com/paper/2602.19153/) (2026)
- [FlowPlace: Flow Matching for Chip Placement](https://mlsanity.com/paper/2604.23658/) (2026)
- [Error-Conditioned Neural Solvers](https://mlsanity.com/paper/2606.27354/) (2026)
- [PosteriorBench: From Point Estimates to Posterior Matching in Evaluating Generative Inverse Solvers](https://mlsanity.com/paper/2609.20794/) (2026)
- [PnP-DA: Towards Principled Plug-and-Play Integration of Variational Data Assimilation and Generative Models](https://mlsanity.com/paper/2508.00325/) (2025)
- [Sparsely-Supervised Data Assimilation via Physics-Informed Schr"odinger Bridge](https://mlsanity.com/paper/2603.22319/) (2026)
- [Enforcing Constraints in Generative Sampling via Adaptive Correction Scheduling](https://mlsanity.com/paper/2605.11214/) (2026)
- [SNAP-FM: Sparse Nonlinear Accelerated Projection for Physics-Constrained Generative Modeling](https://mlsanity.com/paper/2607.00095/) (2026)

## Abstract (arXiv)

Generative models that satisfy hard constraints are critical in many scientific and engineering applications, where physical laws or system requirements must be strictly respected. Many existing constrained generative models, especially those developed for computer vision, rely heavily on gradient information, which is often sparse or computationally expensive in some fields, e.g., partial differential equations (PDEs). In this work, we introduce a novel framework for adapting pre-trained, unconstrained flow-matching models to satisfy constraints exactly in a zero-shot manner without requiring expensive gradient computations or fine-tuning. Our framework, ECI sampling, alternates between extrapolation (E), correction (C), and interpolation (I) stages during each iterative sampling step of flow matching sampling to ensure accurate integration of constraint information while preserving the validity of the generation. We demonstrate the effectiveness of our approach across various PDE systems, showing that ECI-guided generation strictly adheres to physical constraints and accurately captures complex distribution shifts induced by these constraints. Empirical results demonstrate that our framework consistently outperforms baseline approaches in various zero-shot constrained generation tasks and also achieves competitive results in the regression tasks without additional fine-tuning.