# Learning differentiable solvers for systems with hard constraints

arXiv: https://arxiv.org/abs/2207.08675 · HTML: https://mlsanity.com/paper/2207.08675/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 42 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 23 of 23)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers](https://mlsanity.com/paper/2007.00016/) (2020)
- [Efficient and Modular Implicit Differentiation](https://mlsanity.com/paper/2105.15183/) (2021)
- [DC3: A learning method for optimization with hard constraints](https://mlsanity.com/paper/2104.12225/) (2021)
- [Implicit Deep Learning](https://mlsanity.com/paper/1908.06315/) (2019)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [Linearly constrained Gaussian processes](https://mlsanity.com/paper/1703.00787/) (2017)
- [Learning continuous models for continuous physics](https://mlsanity.com/paper/2202.08494/) (2022)
- [On the Differentiability of the Solution to Convex Optimization Problems](https://mlsanity.com/paper/1804.05098/) (2018)
- [Algorithmic Linearly Constrained Gaussian Processes](https://mlsanity.com/paper/1801.09197/) (2018)
- [Linearly Constrained Neural Networks](https://mlsanity.com/paper/2002.01600/) (2020)
- [ANODEV2: A Coupled Neural ODE Evolution Framework](https://mlsanity.com/paper/1906.04596/) (2019)

## Built on by (most cited first, 30 of 34)

- [Learning Physical Models that Can Respect Conservation Laws](https://mlsanity.com/paper/2302.11002/) (2023)
- [AB-UPT: Scaling Neural CFD Surrogates for High-Fidelity Automotive Aerodynamics Simulations via Anchored-Branched Universal Physics Transformers](https://mlsanity.com/paper/2502.09692/) (2025)
- [Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraints](https://mlsanity.com/paper/2506.04171/) (2025)
- [Gradient-Free Generation for Hard-Constrained Systems](https://mlsanity.com/paper/2412.01786/) (2024)
- [Harnessing the Power of Neural Operators with Automatically Encoded Conservation Laws](https://mlsanity.com/paper/2312.11176/) (2023)
- [Scaling physics-informed hard constraints with mixture-of-experts](https://mlsanity.com/paper/2402.13412/) (2024)
- [SuperBench: A Super-Resolution Benchmark Dataset for Scientific Machine Learning](https://mlsanity.com/paper/2306.14070/) (2023)
- [Using Uncertainty Quantification to Characterize and Improve Out-of-Domain Learning for PDEs](https://mlsanity.com/paper/2403.10642/) (2024)
- [AhmedML: High-Fidelity Computational Fluid Dynamics Dataset for Incompressible, Low-Speed Bluff Body Aerodynamics](https://mlsanity.com/paper/2407.20801/) (2024)
- [A New Computationally Simple Approach for Implementing Neural Networks with Output Hard Constraints](https://mlsanity.com/paper/2307.10459/) (2023)
- [Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology](https://mlsanity.com/paper/2310.05227/) (2023)
- [End-to-End Probabilistic Framework for Learning with Hard Constraints](https://mlsanity.com/paper/2506.07003/) (2025)
- [Stability-Aware Training of Machine Learning Force Fields with Differentiable Boltzmann Estimators](https://mlsanity.com/paper/2402.13984/) (2024)
- [Learning Physics for Unveiling Hidden Earthquake Ground Motions via Conditional Generative Modeling](https://mlsanity.com/paper/2407.15089/) (2024)
- [Enabling Automatic Differentiation with Mollified Graph Neural Operators](https://mlsanity.com/paper/2504.08277/) (2025)
- [Strictly Constrained Generative Modeling via Split Augmented Langevin Sampling](https://mlsanity.com/paper/2505.18017/) (2025)
- [EddyFormer: Accelerated Neural Simulations of Three-Dimensional Turbulence at Scale](https://mlsanity.com/paper/2510.24173/) (2025)
- [Neural Fields with Hard Constraints of Arbitrary Differential Order](https://mlsanity.com/paper/2306.08943/) (2023)
- [Gradient-free online learning of subgrid-scale dynamics with neural emulators](https://mlsanity.com/paper/2310.19385/) (2023)
- [Projected Neural Differential Equations for Learning Constrained Dynamics](https://mlsanity.com/paper/2410.23667/) (2024)
- [Adaptive Physics-informed Neural Networks: A Survey](https://mlsanity.com/paper/2503.18181/) (2025)
- [Residuals Are Not Enough: Limits of Physics-Informed Pre-Training for Scientific Foundation Models](https://mlsanity.com/paper/2503.19081/) (2025)
- [Guaranteeing Conservation Laws with Projection in Physics-Informed Neural Networks](https://mlsanity.com/paper/2410.17445/) (2024)
- [CONFIDE: Contextual Finite Differences Modelling of PDEs](https://mlsanity.com/paper/2303.15827/) (2023)
- [Exact and soft boundary conditions in Physics-Informed Neural Networks for the Variable Coefficient Poisson equation](https://mlsanity.com/paper/2310.02548/) (2023)
- [Differentiable Autoencoding Neural Operator for Interpretable and Integrable Latent Space Modeling](https://mlsanity.com/paper/2510.00233/) (2025)
- [Neural DAEs: Constrained neural networks](https://mlsanity.com/paper/2211.14302/) (2022)
- [HoP: Homeomorphic Polar Learning for Hard Constrained Optimization](https://mlsanity.com/paper/2502.00304/) (2025)
- [Physically consistent model learning for reaction-diffusion systems](https://mlsanity.com/paper/2512.14240/) (2025)
- [Constraint-Aware Neurosymbolic Uncertainty Quantification with Bayesian Deep Learning for Scientific Discovery](https://mlsanity.com/paper/2601.12442/) (2026)

## Abstract (arXiv)

We introduce a practical method to enforce partial differential equation (PDE) constraints for functions defined by neural networks (NNs), with a high degree of accuracy and up to a desired tolerance. We develop a differentiable PDE-constrained layer that can be incorporated into any NN architecture. Our method leverages differentiable optimization and the implicit function theorem to effectively enforce physical constraints. Inspired by dictionary learning, our model learns a family of functions, each of which defines a mapping from PDE parameters to PDE solutions. At inference time, the model finds an optimal linear combination of the functions in the learned family by solving a PDE-constrained optimization problem. Our method provides continuous solutions over the domain of interest that accurately satisfy desired physical constraints. Our results show that incorporating hard constraints directly into the NN architecture achieves much lower test error when compared to training on an unconstrained objective.