mlsanityData to 7 Oct 2026

ConFIG: Towards Conflict-free Training of Physics Informed Neural Networks

arXiv 2408.11104 ↗First posted 20 Aug 2024cs.LGcited by 43

Not on the map. Why?

Abstract (arXiv)

The loss functions of many learning problems contain multiple additive terms that can disagree and yield conflicting update directions. For Physics-Informed Neural Networks (PINNs), loss terms on initial/boundary conditions and physics equations are particularly interesting as they are well-established as highly difficult tasks. To improve learning the challenging multi-objective task posed by PINNs, we propose the ConFIG method, which provides conflict-free updates by ensuring a positive dot product between the final update and each loss-specific gradient. It also maintains consistent optimization rates for all loss terms and dynamically adjusts gradient magnitudes based on conflict levels. We additionally leverage momentum to accelerate optimizations by alternating the back-propagation of different loss terms. We provide a mathematical proof showing the convergence of the ConFIG method, and it is evaluated across a range of challenging PINN scenarios. ConFIG consistently shows superior performance and runtime compared to baseline methods. We also test the proposed method in a classic multi-task benchmark, where the ConFIG method likewise exhibits a highly promising performance. Source code is available at https://tum-pbs.github.io/ConFIG

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 20 Aug 2024.

Builds on

32 of 49 references.

References · 32

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  2. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  3. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  4. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  5. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  6. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  7. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
  8. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  9. 2018End-to-End Multi-Task Learning with Attentioncited by 533
  10. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  11. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  12. 2019Orthogonal Gradient Descent for Continual Learningcited by 361
20 more
  1. 2020Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanismcited by 231
  2. 2022Multi-Task Learning as a Bargaining Gamecited by 209
  3. 2020Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropoutcited by 199
  4. 2020Learning to Control PDEs with Differentiable Physicscited by 131
  5. 2020Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networkscited by 122
  6. 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 100
  7. 2020Continual Learning in Low-rank Orthogonal Subspacescited by 99
  8. 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 98
  9. 2021Multi-Objective Loss Balancing for Physics-Informed Deep Learningcited by 86
  10. 2023PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEscited by 84
  11. 2021RotoGrad: Gradient Homogenization in Multitask Learningcited by 82
  12. 2022Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Samplingcited by 71
  13. 2020An overview on deep learning-based approximation methods for partial differential equationscited by 65
  14. 2021Learning in Sinusoidal Spaces with Physics-Informed Neural Networkscited by 61
  15. 2020A Dual-Dimer Method for Training Physics-Constrained Neural Networks with Minimax Architecturecited by 50
  16. 2022Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs)cited by 35
  17. 2021Recipes for when Physics Fails: Recovering Robust Learning of Physics Informed Neural Networkscited by 18
  18. 2023MultiAdam: Parameter-wise Scale-invariant Optimizer for Multiscale Training of Physics-informed Neural Networkscited by 17
  19. 2023GDOD: Effective Gradient Descent using Orthogonal Decomposition for Multi-Task Learningcited by 6
  20. 2021A generic physics-informed neural network-based framework for reliability assessment of multi-state systemscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 16; 2026: 26.

43 papers.

Sort:

By area:

  1. 2025Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspectivecited by 55
  2. 2025Physics vs Distributions: Pareto Optimal Flow Matching with Physics Constraintscited by 26
  3. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 15
  4. 2025Guiding diffusion models to reconstruct flow fields from sparse datacited by 11
  5. 2025QCPINN: Quantum-Classical Physics-Informed Neural Networks for Solving PDEscited by 9
  6. 2025Physics-Constrained Fine-Tuning of Flow-Matching Models for Generation and Inverse Problemscited by 8
  7. 2025Hard-constraining Neumann boundary conditions in physics-informed neural networks via Fourier feature embeddingscited by 5
  8. 2025A Control Perspective on Training PINNscited by 3
  9. 2025Neural Tangent Kernel of Neural Networks with Loss Informed by Differential Operatorscited by 3
  10. 2025Sobolev Training of End-to-End Optimization Proxiescited by 2
  11. 2025Leveraging Influence Functions for Resampling Data in Physics-Informed Neural Networkscited by 2
  12. 2025Revisiting Heat Flux Analysis of Tungsten Monoblock Divertor on EAST using Physics-Informed Neural Networkcited by 2
31 more
  1. 2026Tackling Failure Modes of PINNs and PIKANs Using Conflict-Free Gradientscited by 2
  2. 2024PINNfluence: Interpreting PINNs through Influence Functionscited by 1
  3. 2026Per-Loss Adapters for Gradient Conflict in Physics-Informed Neural Networkscited by 1
  4. 2026On the training of physics-informed neural operators for solving parametric partial differential equationscited by 1
  5. 2025Modelling of Underwater Vehicles using Physics-Informed Neural Networks with Controlcited by 0
  6. 2025Mask-PINNs: Mitigating Internal Covariate Shift in Physics-Informed Neural Networkscited by 0
  7. 2025Physics-based machine learning for mantle convection simulationscited by 0
  8. 2025AutoBalance: An Automatic Balancing Framework for Training Physics-Informed Neural Networkscited by 0
  9. 2026Geometry-Aware Physics-Informed PointNets for Modeling Flows Across Porous Structurescited by 0
  10. 2026Efficient RGB-D Scene Understanding via Multi-task Adaptive Learning and Cross-dimensional Feature Guidancecited by 0
  11. 2026PhysSkin: Real-Time and Generalizable Physics-Based Animation via Self-Supervised Neural Skinningcited by 0
  12. 2026Resolving gradient pathology in physics-informed epidemiological modelscited by 0
  13. 2026Towards a Foundation-Model Paradigm for Aerodynamic Prediction in Three-dimensional Designcited by 0
  14. 2026Conflict-Aware Harmonized Rotational Gradient for Multiscale Kinetic Regimescited by 0
  15. 2026PILIR: Physics-Informed Local Implicit Representationcited by 0
  16. 2026Chebyshev Center-Based Direction Selection for Multi-Objective Optimization and Training PINNscited by 0
  17. 2026MetaColloc: Optimization-Free PDE Solving via Meta-Learned Basis Functionscited by 0
  18. 2026Eradicating Negative Transfer in Multi-Physics Foundation Models via Sparse Mixture-of-Experts Routingcited by 0
  19. 2026Curriculum Learning of Physics-Informed Neural Networks based on Spatial Correlationcited by 0
  20. 2026CRAFT: Conflict-Resolved Aggregation for Federated Trainingcited by 0
  21. 2026Mitigating Gradient Pathology in PINNs through Aligned Constraintcited by 0
  22. 2026A Unified Framework for Gradient Aggregation in Multi-Objective Optimizationcited by 0
  23. 2026Effective Dimensionality as an Operator Invariant for Physics-Preserving Constraint Adaptation in Physics-Informed Neural Networkscited by 0
  24. 2026Modularity-Free Conflict-Averse Training for Generalized PINNscited by 0
  25. 2026Finite basis physics-informed neural networks with hard constraints for viscous fluid flow in highly perforated domainscited by 0
  26. 2026Beyond Field Accuracy: Two-Axis Diagnosis of Inverse-PINN Parameter Errorcited by 0
  27. 2026Gradient-Update Mismatch: Rethinking Conflict-Free Training of Physics-Informed Neural Networkscited by 0
  28. 2026From Trainability Diagnostics to Optimization Claims: Boundaries and Controls in Variational Quantum Optimizationcited by 0
  29. 2026Periodic Neural Mapping for Unsteady Rotor-Blade Pressure and Aeroelastic Load Predictioncited by 0
  30. 2026Gradient Surgery for Physics-Informed Neural Networkscited by 0
  31. 2026PMosFM: Preconditioned Manifold Matching for One-Step Physics-Constrained Generationcited by 0