# Characterizing and Mitigating Distributional Bias in Physics-Informed Generative Models

arXiv: https://arxiv.org/abs/2610.16582 · HTML: https://mlsanity.com/paper/2610.16582/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 31 of its 56 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 30 of 31)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Structure-based Drug Design with Equivariant Diffusion Models](https://mlsanity.com/paper/2210.13695/) (2022)
- [A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstruction](https://mlsanity.com/paper/2211.14680/) (2022)
- [SE(3)-Stochastic Flow Matching for Protein Backbone Generation](https://mlsanity.com/paper/2310.02391/) (2023)
- [DiffusionPDE: Generative PDE-Solving Under Partial Observation](https://mlsanity.com/paper/2406.17763/) (2024)
- [Physics-Informed Diffusion Models](https://mlsanity.com/paper/2403.14404/) (2024)
- [CoCoGen: Physically-Consistent and Conditioned Score-based Generative Models for Forward and Inverse Problems](https://mlsanity.com/paper/2312.10527/) (2023)
- [Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraints](https://mlsanity.com/paper/2506.04171/) (2025)
- [ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks](https://mlsanity.com/paper/2408.11104/) (2024)
- [Mirror Diffusion Models for Constrained and Watermarked Generation](https://mlsanity.com/paper/2310.01236/) (2023)
- [Gradient-Free Generation for Hard-Constrained Systems](https://mlsanity.com/paper/2412.01786/) (2024)
- [Synthetic Lagrangian Turbulence by Generative Diffusion Models](https://mlsanity.com/paper/2307.08529/) (2023)
- [A probabilistic generative model for semi-supervised training of coarse-grained surrogates and enforcing physical constraints through virtual observables](https://mlsanity.com/paper/2006.01789/) (2020)
- [Physics vs Distributions: Pareto Optimal Flow Matching with Physics Constraints](https://mlsanity.com/paper/2506.08604/) (2025)
- [Uncertainty-aware Surrogate Models for Airfoil Flow Simulations with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2312.05320/) (2023)
- [Precipitation Downscaling with Spatiotemporal Video Diffusion](https://mlsanity.com/paper/2312.06071/) (2023)
- [Solving Inverse Physics Problems with Score Matching](https://mlsanity.com/paper/2301.10250/) (2023)
- [HIP: Hessian Interatomic Potentials without derivatives](https://mlsanity.com/paper/2509.21624/) (2025)
- [Physics-Constrained Fine-Tuning of Flow-Matching Models for Generation and Inverse Problems](https://mlsanity.com/paper/2508.09156/) (2025)
- [Neural Approximate Mirror Maps for Constrained Diffusion Models](https://mlsanity.com/paper/2406.12816/) (2024)
- [Rao-Blackwell Gradient Estimators for Equivariant Denoising Diffusion](https://mlsanity.com/paper/2502.09890/) (2025)
- [Physics-informed diffusion models in spectral space](https://mlsanity.com/paper/2602.09708/) (2026)
- [Calibrated Test-Time Guidance for Bayesian Inference](https://mlsanity.com/paper/2602.22428/) (2026)
- [PIRF: Physics-Informed Reward Fine-Tuning for Diffusion Models](https://mlsanity.com/paper/2509.20570/) (2025)
- [Ultra Fast PDE Solving via Physics Guided Few-step Diffusion](https://mlsanity.com/paper/2602.03627/) (2026)
- [Stabilizing Physics-Informed Consistency Models via Structure-Preserving Training](https://mlsanity.com/paper/2602.09303/) (2026)
- [The Right Measure for Physics-Constrained Generation: A Co-Area Correction for Posterior-Consistent PDE Inverse Problems](https://mlsanity.com/paper/2606.04804/) (2026)

## Abstract (arXiv)

Physics-informed generative models augment a data-fitting objective with residual penalties that encourage physical constraints to hold in generated samples. However, improving constraint satisfaction need not improve fidelity to the data distribution. We study this conflict in residual-regularized diffusion models and identify two possible consequences. First, the same prediction error can incur different residual costs near different feasible states, preferring states with lower residual sensitivity (avoidance). Second, this preference can extend to feasible states outside the data support, where the residual also vanishes (leakage). Motivated by this analysis, we propose a data-anchored penalty that weights prediction errors by the residual Jacobian at the clean training sample and normalizes the resulting magnitude. This removes the incentive to relocate to less sensitive states without changing the model architecture or sampling procedure. Experiments on a controlled synthetic system, three PDE systems, and molecular transition-state generation with flow matching demonstrate the benefits of anchoring. Code is available at https://github.com/TanjaBien/anchored_diffusion.