# Training neural networks under physical constraints using a stochastic augmented Lagrangian approach

arXiv: https://arxiv.org/abs/2009.07330 · HTML: https://mlsanity.com/paper/2009.07330/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2020.

## Built on by (most cited first, 20 of 20)

- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Neural Networks with Physics-Informed Architectures and Constraints for Dynamical Systems Modeling](https://mlsanity.com/paper/2109.06407/) (2021)
- [Physics and Equality Constrained Artificial Neural Networks: Application to Forward and Inverse Problems with Multi-fidelity Data Fusion](https://mlsanity.com/paper/2109.14860/) (2021)
- [HardNet: Hard-Constrained Neural Networks with Universal Approximation Guarantees](https://mlsanity.com/paper/2410.10807/) (2024)
- [Physics-Informed Neural Networks with Hard Linear Equality Constraints](https://mlsanity.com/paper/2402.07251/) (2024)
- [An adaptive augmented Lagrangian method for training physics and equality constrained artificial neural networks](https://mlsanity.com/paper/2306.04904/) (2023)
- [Complexity of Single Loop Algorithms for Nonlinear Programming with Stochastic Objective and Constraints](https://mlsanity.com/paper/2311.00678/) (2023)
- [NCVX: A General-Purpose Optimization Solver for Constrained Machine and Deep Learning](https://mlsanity.com/paper/2210.00973/) (2022)
- [Invariant preservation in machine learned PDE solvers via error correction](https://mlsanity.com/paper/2303.16110/) (2023)
- [Guaranteed Conformance of Neurosymbolic Models to Natural Constraints](https://mlsanity.com/paper/2212.01346/) (2022)
- [Stochastic Smoothed Primal-Dual Algorithms for Nonconvex Optimization with Linear Inequality Constraints](https://mlsanity.com/paper/2504.07607/) (2025)
- [Achieving Constraints in Neural Networks: A Stochastic Augmented Lagrangian Approach](https://mlsanity.com/paper/2310.16647/) (2023)
- [Physics-Informed Neural Networks with Trust-Region Sequential Quadratic Programming](https://mlsanity.com/paper/2409.10777/) (2024)
- [PINNverse: Accurate parameter estimation in differential equations from noisy data with constrained physics-informed neural networks](https://mlsanity.com/paper/2504.05248/) (2025)
- [Conditionally adaptive augmented Lagrangian method for physics-informed learning of forward and inverse problems](https://mlsanity.com/paper/2508.15695/) (2025)
- [Adjoint-Matching Neural Network Surrogates for Fast 4D-Var Data Assimilation](https://mlsanity.com/paper/2111.08626/) (2021)
- [Multi-Objective Physics-Guided Recurrent Neural Networks for Identifying Non-Autonomous Dynamical Systems](https://mlsanity.com/paper/2204.12972/) (2022)
- [SnareNet: Flexible Repair Layers for Neural Networks with Hard Constraints](https://mlsanity.com/paper/2602.09317/) (2026)
- [Sample-wise Constrained Learning via a Sequential Penalty Approach with Applications in Image Processing](https://mlsanity.com/paper/2601.16812/) (2026)
- [ORACLE: Optimizer-Relative Alignment for Constrained LEarning](https://mlsanity.com/paper/2610.09040/) (2026)

## Abstract (arXiv)

We investigate the physics-constrained training of an encoder-decoder neural network for approximating the Fokker-Planck-Landau collision operator in the 5-dimensional kinetic fusion simulation in XGC. To train this network, we propose a stochastic augmented Lagrangian approach that utilizes pyTorch's native stochastic gradient descent method to solve the inner unconstrained minimization subproblem, paired with a heuristic update for the penalty factor and Lagrange multipliers in the outer augmented Lagrangian loop. Our training results for a single ion species case, with self-collisions and collision against electrons, show that the proposed stochastic augmented Lagrangian approach can achieve higher model prediction accuracy than training with a fixed penalty method for our application problem, with the accuracy high enough for practical applications in kinetic simulations.