mlsanityData to 14 Oct 2026

Stacked networks improve physics-informed training: applications to neural networks and deep operator networks

arXiv 2311.06483 ↗First posted 11 Nov 2023cs.LG cs.NA math.NAcited by 21

Not on the map. Why?

Abstract (arXiv)

Physics-informed neural networks and operator networks have shown promise for effectively solving equations modeling physical systems. However, these networks can be difficult or impossible to train accurately for some systems of equations. We present a novel multifidelity framework for stacking physics-informed neural networks and operator networks that facilitates training. We successively build a chain of networks, where the output at one step can act as a low-fidelity input for training the next step, gradually increasing the expressivity of the learned model. The equations imposed at each step of the iterative process can be the same or different (akin to simulated annealing). The iterative (stacking) nature of the proposed method allows us to progressively learn features of a solution that are hard to learn directly. Through benchmark problems including a nonlinear pendulum, the wave equation, and the viscous Burgers equation, we show how stacking can be used to improve the accuracy and reduce the required size of physics-informed neural networks and operator networks.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 47 of its 80 references that are in the map. First posted on arXiv 11 Nov 2023.

Builds on

47 of 80 references.

References · 47

  1. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  2. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  3. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  4. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  5. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 734
  6. 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
  7. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
  8. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 608
  9. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
  10. 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 540
  11. 2021Physics-informed neural networks (PINNs) for fluid mechanics: A reviewcited by 473
  12. 2021U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flowcited by 297
35 more
  1. 2022A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networkscited by 236
  2. 2020Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanismcited by 233
  3. 2022Respecting causality is all you need for training physics-informed neural networkscited by 156
  4. 2022A Physics-informed Diffusion Model for High-fidelity Flow Field Reconstructioncited by 152
  5. 2021Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equationscited by 145
  6. 2020Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networkscited by 123
  7. 2021Efficient training of physics-informed neural networks via importance samplingcited by 110
  8. 2021A physics-informed variational DeepONet for predicting the crack path in brittle materialscited by 98
  9. 2022Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transportcited by 92
  10. 2021Long-time integration of parametric evolution equations with physics-informed DeepONetscited by 89
  11. 2021Improved architectures and training algorithms for deep operator networkscited by 86
  12. 2022Deep transfer operator learning for partial differential equations under conditional shiftcited by 83
  13. 2019Physics-Informed Neural Networks for Power Systemscited by 80
  14. 2022Mitigating Propagation Failures in Physics-informed Neural Networks using Retain-Resample-Release (R3) Samplingcited by 73
  15. 2022Failure-informed adaptive sampling for PINNscited by 64
  16. 2022Multifidelity Deep Operator Networks For Data-Driven and Physics-Informed Problemscited by 58
  17. 2022Interfacing Finite Elements with Deep Neural Operators for Fast Multiscale Modeling of Mechanics Problemscited by 48
  18. 2023A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositionscited by 48
  19. 2021DAE-PINN: A Physics-Informed Neural Network Model for Simulating Differential-Algebraic Equations with Application to Power Networkscited by 28
  20. 2022Transfer Learning Enhanced DeepONet for Long-Time Prediction of Evolution Equationscited by 28
  21. 2022Bi-fidelity Modeling of Uncertain and Partially Unknown Systems using DeepONetscited by 27
  22. 2022Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networkscited by 26
  23. 2021Multifidelity Modeling for Physics-Informed Neural Networks (PINNs)cited by 25
  24. 2021Galerkin Neural Networks: A Framework for Approximating Variational Equations with Error Controlcited by 24
  25. 2022On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networkscited by 23
  26. 2022Experience report of physics-informed neural networks in fluid simulations: pitfalls and frustrationcited by 21
  27. 2021Recipes for when Physics Fails: Recovering Robust Learning of Physics Informed Neural Networkscited by 18
  28. 2021Applying Machine Learning to Study Fluid Mechanicscited by 13
  29. 2023A multifidelity approach to continual learning for physical systemscited by 13
  30. 2023Predictive Limitations of Physics-Informed Neural Networks in Vortex Sheddingcited by 12
  31. 2023A Hybrid Deep Neural Operator/Finite Element Method for Ice-Sheet Modelingcited by 11
  32. 2023Feature-adjacent multi-fidelity physics-informed machine learning for partial differential equationscited by 8
  33. 2022A Data-driven Multi-fidelity Physics-informed Learning Framework for Smart Manufacturing: A Composites Processing Case Studycited by 7
  34. 2023Efficient kernel surrogates for neural network-based regressioncited by 7
  35. 2023Machine learning of hidden variables in multiscale fluid simulationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 11; 2025: 4; 2026: 5.

21 papers.

Sort:

By area:

  1. 2024From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learningcited by 73
  2. 2024Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problemscited by 35
  3. 2023Machine learning and domain decomposition methods -- a surveycited by 14
  4. 2024SPIKANs: Separable Physics-Informed Kolmogorov-Arnold Networkscited by 12
  5. 2024Ensemble and Mixture-of-Experts DeepONets For Operator Learningcited by 9
  6. 2024Multifidelity Kolmogorov-Arnold Networkscited by 9
  7. 2024Multifidelity domain decomposition-based physics-informed neural networks and operators for time-dependent problemscited by 4
  8. 2024Quantifying Training Difficulty and Accelerating Convergence in Neural Network-Based PDE Solverscited by 4
  9. 2025Iterative Training of Physics-Informed Neural Networks with Fourier-enhanced Featurescited by 4
  10. 2024What do physics-informed DeepONets learn? Understanding and improving training for scientific computing applicationscited by 3
  11. 2025mLaSDI: Multi-stage latent space dynamics identificationcited by 2
  12. 2025Sequential decoder training for improved latent space dynamics identificationcited by 2
9 more
  1. 2024Deep Learning without Global Optimization by Random Fourier Neural Networkscited by 1
  2. 2024Deep operator network models for predicting post-burn contractioncited by 1
  3. 2025Bridging quantum and classical computing for partial differential equations through multifidelity machine learningcited by 1
  4. 2024Two-scale Neural Networks for Partial Differential Equations with Small Parameterscited by 0
  5. 2026Multi-Fidelity Physics-Informed Neural Networks with Bayesian Uncertainty Quantification and Adaptive Residual Learning for Efficient Solution of Parametric Partial Differential Equationscited by 0
  6. 2026Curriculum-Learned Vanishing Stacked Residual PINNs for Hyperbolic PDE State Reconstructioncited by 0
  7. 2026Scientific Knowledge-Guided Machine Learning for Vessel Power Prediction: A Comparative Studycited by 0
  8. 2026Improving the accuracy of physics-informed neural networks via last-layer retrainingcited by 0
  9. 2026Staged Depth Training: A Representation Curriculum for PINNscited by 0