mlsanityData to 7 Oct 2026

Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networks

arXiv 2107.00940 ↗First posted 2 Jul 2021cs.LG cs.NA math.NAcited by 43

Not on the map. Why?

Abstract (arXiv)

We characterize and remedy a failure mode that may arise from multi-scale dynamics with scale imbalances during training of deep neural networks, such as Physics Informed Neural Networks (PINNs). PINNs are popular machine-learning templates that allow for seamless integration of physical equation models with data. Their training amounts to solving an optimization problem over a weighted sum of data-fidelity and equation-fidelity objectives. Conflicts between objectives can arise from scale imbalances, heteroscedasticity in the data, stiffness of the physical equation, or from catastrophic interference during sequential training. We explain the training pathology arising from this and propose a simple yet effective inverse-Dirichlet weighting strategy to alleviate the issue. We compare with Sobolev training of neural networks, providing the baseline of analytically boldsymbolε-optimal training. We demonstrate the effectiveness of inverse-Dirichlet weighting in various applications, including a multi-scale model of active turbulence, where we show orders of magnitude improvement in accuracy and convergence over conventional PINN training. For inverse modeling using sequential training, we find that inverse-Dirichlet weighting protects a PINN against catastrophic forgetting.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 28 of its 58 references that are in the map. First posted on arXiv 2 Jul 2021.

Builds on

28 of 58 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2021Convex optimizationcited by 2,479
  5. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  6. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  7. 2018On the Spectral Bias of Neural Networkscited by 1,321
  8. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  9. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 809
  10. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  11. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  12. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
16 more
  1. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 360
  2. 2020B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Datacited by 329
  3. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 298
  4. 2018Training behavior of deep neural network in frequency domaincited by 174
  5. 2017Sobolev Training for Neural Networkscited by 136
  6. 2020Understanding and mitigating gradient pathologies in physics-informed neural networkscited by 134
  7. 2020Descending through a Crowded Valley - Benchmarking Deep Learning Optimizerscited by 118
  8. 2019Error bounds for approximations with deep ReLU neural networks in Ws,p normscited by 95
  9. 2020Physics informed deep learning for computational elastodynamics without labeled datacited by 79
  10. 2018Deep Learning of Vortex Induced Vibrationscited by 74
  11. 2019DeepMoD: Deep learning for Model Discovery in noisy datacited by 70
  12. 2019Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networkscited by 63
  13. 2018Understanding training and generalization in deep learning by Fourier analysiscited by 43
  14. 2020DenoiSeg: Joint Denoising and Segmentationcited by 20
  15. 2018Deep Learning of Turbulent Scalar Mixingcited by 17
  16. 2019Physics-informed semantic inpainting: Application to geostatistical modelingcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 10; 2024: 11; 2025: 11; 2026: 5.

43 papers.

Sort:

By area:

  1. 2022Respecting causality is all you need for training physics-informed neural networkscited by 154
  2. 2023An Expert's Guide to Training Physics-informed Neural Networkscited by 119
  3. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 88
  4. 2023PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEscited by 84
  5. 2024PirateNets: Physics-informed Deep Learning with Residual Adaptive Networkscited by 70
  6. 2024RoPINN: Region Optimized Physics-Informed Neural Networkscited by 26
  7. 2022On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networkscited by 23
  8. 2024Functional Tensor Decompositions for Physics-Informed Neural Networkscited by 11
  9. 2024BiLO: Bilevel Local Operator Learning for PDE Inverse Problemscited by 8
  10. 2023Personalized Predictions of Glioblastoma Infiltration: Mathematical Models, Physics-Informed Neural Networks and Multimodal Scanscited by 7
  11. 2023PINN Training using Biobjective Optimization: The Trade-off between Data Loss and Residual Losscited by 6
  12. 2024Physics-Informed Neural Networks for Dynamic Process Operations with Limited Physical Knowledge and Datacited by 6
31 more
  1. 2024A physics-informed transformer neural operator for learning generalized solutions of initial boundary value problemscited by 6
  2. 2023Stochastic force inference via density estimationcited by 5
  3. 2025Dual-Balancing for Physics-Informed Neural Networkscited by 5
  4. 2022Tunable Complexity Benchmarks for Evaluating Physics-Informed Neural Networks on Coupled Ordinary Differential Equationscited by 4
  5. 2022Replacing Automatic Differentiation by Sobolev Cubatures fastens Physics Informed Neural Nets and strengthens their Approximation Powercited by 4
  6. 2023LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometrycited by 4
  7. 2024Adapting Physics-Informed Neural Networks to Improve ODE Optimization in Mosquito Population Dynamicscited by 4
  8. 2024Solving Differential Equations with Constrained Learningcited by 3
  9. 2025A Control Perspective on Training PINNscited by 3
  10. 2022Physics-Informed Koopman Networkcited by 2
  11. 2024Common pitfalls to avoid while using multiobjective optimization in machine learningcited by 2
  12. 2023Learning Partial Differential Equations by Spectral Approximates of General Sobolev Spacescited by 1
  13. 2023Maximum-likelihood Estimators in Physics-Informed Neural Networks for High-dimensional Inverse Problemscited by 1
  14. 2023Auto-weighted Bayesian Physics-Informed Neural Networks and robust estimations for multitask inverse problems in pore-scale imaging of dissolutioncited by 1
  15. 2025Sample-Efficient Reinforcement Learning of Koopman eNMPCcited by 1
  16. 2025Solving 2-D Helmholtz equation in the rectangular, circular, and elliptical domains using neural networkscited by 1
  17. 2025Physics-Informed Neural Networks For Semiconductor Film Deposition: A Reviewcited by 1
  18. 2025Enhancing Stability of Physics-Informed Neural Network Training Through Saddle-Point Reformulationcited by 1
  19. 2025RamPINN: Recovering Raman Spectra From Coherent Anti-Stokes Spectra Using Embedded Physicscited by 1
  20. 2025Graph Network-based Structural Simulator: Graph Neural Networks for Structural Dynamicscited by 1
  21. 2026AdamFLIP: Adaptive Momentum Feedback Linearization Optimization for Hard Constrained PINN Trainingcited by 1
  22. 2023Learning locally dominant force balances in active particle systemscited by 0
  23. 2024Two-scale Neural Networks for Partial Differential Equations with Small Parameterscited by 0
  24. 2024Equation identification for fluid flows via physics-informed neural networkscited by 0
  25. 2025Exploration of Hepatitis B Virus Infection Dynamics through Physics-Informed Deep Learning Approachcited by 0
  26. 2025Reliable and efficient inverse analysis using physics-informed neural networks with normalized distance functions and adaptive weight tuningcited by 0
  27. 2025Prediction of acoustic field in 1-D uniform duct with varying mean flow and temperature using neural networkscited by 0
  28. 2026Estimating Dense-Packed Zone Height in Liquid-Liquid Separation: A Physics-Informed Neural Network Approachcited by 0
  29. 2026MUSA-PINN: Multi-scale Weak-form Physics-Informed Neural Networks for Fluid Flow in Complex Geometriescited by 0
  30. 2026Learning biophysical models of gene regulation with probability flow matchingcited by 0
  31. 2026Differentiable Chemistry in PINNs for Solving Parameterized and Stiff Reaction Systemscited by 0