# ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks

arXiv: https://arxiv.org/abs/2408.11104 · HTML: https://mlsanity.com/paper/2408.11104/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 49 references that are in the map.
- First posted on arXiv 20 Aug 2024.

## Builds on (most cited first, 30 of 32)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Conflict-Averse Gradient Descent for Multi-task Learning](https://mlsanity.com/paper/2110.14048/) (2021)
- [Orthogonal Gradient Descent for Continual Learning](https://mlsanity.com/paper/1910.07104/) (2019)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Multi-Task Learning as a Bargaining Game](https://mlsanity.com/paper/2202.01017/) (2022)
- [Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout](https://mlsanity.com/paper/2010.06808/) (2020)
- [Learning to Control PDEs with Differentiable Physics](https://mlsanity.com/paper/2001.07457/) (2020)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning](https://mlsanity.com/paper/2111.10603/) (2021)
- [Adapting Auxiliary Losses Using Gradient Similarity](https://mlsanity.com/paper/1812.02224/) (2018)
- [Continual Learning in Low-rank Orthogonal Subspaces](https://mlsanity.com/paper/2010.11635/) (2020)
- [Multi-Objective Loss Balancing for Physics-Informed Deep Learning](https://mlsanity.com/paper/2110.09813/) (2021)
- [PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs](https://mlsanity.com/paper/2306.08827/) (2023)
- [RotoGrad: Gradient Homogenization in Multitask Learning](https://mlsanity.com/paper/2103.02631/) (2021)
- [Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Sampling](https://mlsanity.com/paper/2207.02338/) (2022)
- [An overview on deep learning-based approximation methods for partial differential equations](https://mlsanity.com/paper/2012.12348/) (2020)
- [Learning in Sinusoidal Spaces with Physics-Informed Neural Networks](https://mlsanity.com/paper/2109.09338/) (2021)
- [A Dual-Dimer Method for Training Physics-Constrained Neural Networks with Minimax Architecture](https://mlsanity.com/paper/2005.00615/) (2020)
- [Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs)](https://mlsanity.com/paper/2205.01059/) (2022)
- [Recipes for when Physics Fails: Recovering Robust Learning of Physics Informed Neural Networks](https://mlsanity.com/paper/2110.13330/) (2021)
- [MultiAdam: Parameter-wise Scale-invariant Optimizer for Multiscale Training of Physics-informed Neural Networks](https://mlsanity.com/paper/2306.02816/) (2023)

## Built on by (most cited first, 30 of 46)

- [Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspective](https://mlsanity.com/paper/2502.00604/) (2025)
- [Physics vs Distributions: Pareto Optimal Flow Matching with Physics Constraints](https://mlsanity.com/paper/2506.08604/) (2025)
- [Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond](https://mlsanity.com/paper/2501.10945/) (2025)
- [Guiding diffusion models to reconstruct flow fields from sparse data](https://mlsanity.com/paper/2510.19971/) (2025)
- [Physics-Constrained Fine-Tuning of Flow-Matching Models for Generation and Inverse Problems](https://mlsanity.com/paper/2508.09156/) (2025)
- [QCPINN: Quantum-Classical Physics-Informed Neural Networks for Solving PDEs](https://mlsanity.com/paper/2503.16678/) (2025)
- [Hard-constraining Neumann boundary conditions in physics-informed neural networks via Fourier feature embeddings](https://mlsanity.com/paper/2504.01093/) (2025)
- [Neural Tangent Kernel of Neural Networks with Loss Informed by Differential Operators](https://mlsanity.com/paper/2503.11029/) (2025)
- [A Control Perspective on Training PINNs](https://mlsanity.com/paper/2501.18582/) (2025)
- [Sobolev Training of End-to-End Optimization Proxies](https://mlsanity.com/paper/2505.11342/) (2025)
- [Leveraging Influence Functions for Resampling Data in Physics-Informed Neural Networks](https://mlsanity.com/paper/2506.16443/) (2025)
- [Revisiting Heat Flux Analysis of Tungsten Monoblock Divertor on EAST using Physics-Informed Neural Network](https://mlsanity.com/paper/2508.03776/) (2025)
- [When PINNs Go Wrong: Pseudo-Time Stepping Against Spurious Solutions](https://mlsanity.com/paper/2604.23528/) (2026)
- [Tackling Failure Modes of PINNs and PIKANs Using Conflict-Free Gradients](https://mlsanity.com/paper/2609.14841/) (2026)
- [PINNfluence: Interpreting PINNs through Influence Functions](https://mlsanity.com/paper/2409.08958/) (2024)
- [Per-Loss Adapters for Gradient Conflict in Physics-Informed Neural Networks](https://mlsanity.com/paper/2605.10136/) (2026)
- [CRAFT: Conflict-Resolved Aggregation for Federated Training](https://mlsanity.com/paper/2605.21317/) (2026)
- [On the training of physics-informed neural operators for solving parametric partial differential equations](https://mlsanity.com/paper/2606.06164/) (2026)
- [Gradient-Update Mismatch: Rethinking Conflict-Free Training of Physics-Informed Neural Networks](https://mlsanity.com/paper/2609.01558/) (2026)
- [Modelling of Underwater Vehicles using Physics-Informed Neural Networks with Control](https://mlsanity.com/paper/2504.20019/) (2025)
- [Mask-PINNs: Mitigating Internal Covariate Shift in Physics-Informed Neural Networks](https://mlsanity.com/paper/2505.06331/) (2025)
- [Physics-based machine learning for mantle convection simulations](https://mlsanity.com/paper/2505.16041/) (2025)
- [AutoBalance: An Automatic Balancing Framework for Training Physics-Informed Neural Networks](https://mlsanity.com/paper/2510.06684/) (2025)
- [Geometry-Aware Physics-Informed PointNets for Modeling Flows Across Porous Structures](https://mlsanity.com/paper/2602.14108/) (2026)
- [Efficient RGB-D Scene Understanding via Multi-task Adaptive Learning and Cross-dimensional Feature Guidance](https://mlsanity.com/paper/2603.07570/) (2026)
- [PhysSkin: Real-Time and Generalizable Physics-Based Animation via Self-Supervised Neural Skinning](https://mlsanity.com/paper/2603.23194/) (2026)
- [Resolving gradient pathology in physics-informed epidemiological models](https://mlsanity.com/paper/2603.23799/) (2026)
- [Towards a Foundation-Model Paradigm for Aerodynamic Prediction in Three-dimensional Design](https://mlsanity.com/paper/2604.18062/) (2026)
- [Conflict-Aware Harmonized Rotational Gradient for Multiscale Kinetic Regimes](https://mlsanity.com/paper/2604.24745/) (2026)
- [PILIR: Physics-Informed Local Implicit Representation](https://mlsanity.com/paper/2605.00385/) (2026)

## Abstract (arXiv)

The loss functions of many learning problems contain multiple additive terms that can disagree and yield conflicting update directions. For Physics-Informed Neural Networks (PINNs), loss terms on initial/boundary conditions and physics equations are particularly interesting as they are well-established as highly difficult tasks. To improve learning the challenging multi-objective task posed by PINNs, we propose the ConFIG method, which provides conflict-free updates by ensuring a positive dot product between the final update and each loss-specific gradient. It also maintains consistent optimization rates for all loss terms and dynamically adjusts gradient magnitudes based on conflict levels. We additionally leverage momentum to accelerate optimizations by alternating the back-propagation of different loss terms. We provide a mathematical proof showing the convergence of the ConFIG method, and it is evaluated across a range of challenging PINN scenarios. ConFIG consistently shows superior performance and runtime compared to baseline methods. We also test the proposed method in a classic multi-task benchmark, where the ConFIG method likewise exhibits a highly promising performance. Source code is available at https://tum-pbs.github.io/ConFIG