mlsanityData to 14 Oct 2026

Improving physics-informed neural networks with meta-learned optimization

arXiv 2303.07127 ↗First posted 13 Mar 2023cs.LG cs.NA math.NAcited by 19

Not on the map. Why?

Abstract (arXiv)

We show that the error achievable using physics-informed neural networks for solving systems of differential equations can be substantially reduced when these networks are trained using meta-learned optimization methods rather than to using fixed, hand-crafted optimizers as traditionally done. We choose a learnable optimization method based on a shallow multi-layer perceptron that is meta-trained for specific classes of differential equations. We illustrate meta-trained optimizers for several equations of practical relevance in mathematical physics, including the linear advection equation, Poisson's equation, the Korteweg--de Vries equation and Burgers' equation. We also illustrate that meta-learned optimizers exhibit transfer learning abilities, in that a meta-trained optimizer on one differential equation can also be successfully deployed on another differential equation.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 13 Mar 2023.

Builds on

27 of 42 references.

References · 27

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  4. 2018On the Spectral Bias of Neural Networkscited by 1,341
  5. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  6. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  7. 2015Automatic differentiation in machine learning: a surveycited by 855
  8. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  9. 2016Learning to learn by gradient descent by gradient descentcited by 620
  10. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 608
  11. 2020hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decompositioncited by 236
  12. 2022A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networkscited by 236
15 more
  1. 2021Learning to Optimize: A Primer and A Benchmarkcited by 208
  2. 2022Respecting causality is all you need for training physics-informed neural networkscited by 156
  3. 2017Learned Optimizers that Scale and Generalizecited by 110
  4. 2019On Empirical Comparisons of Optimizers for Deep Learningcited by 103
  5. 2021Long-time integration of parametric evolution equations with physics-informed DeepONetscited by 89
  6. 2021Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategiescited by 52
  7. 2021Meta-learning PINN loss functionscited by 51
  8. 2017Learning Gradient Descent: Better Generalization and Longer Horizonscited by 50
  9. 2023A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositionscited by 48
  10. 2018Aggregated Momentum: Stability Through Passive Dampingcited by 36
  11. 2021A novel meta-learning initialization method for physics-informed neural networkscited by 35
  12. 2021Physics-informed neural networks for the shallow-water equations on the spherecited by 32
  13. 2022Practical tradeoffs between memory, compute, and performance in learned optimizerscited by 32
  14. 2022A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biasescited by 30
  15. 2020Reverse engineering learned optimizers reveals known and novel mechanismscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 4; 2025: 8; 2026: 3.

19 papers.

Sort:

  1. 2024Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?cited by 31
  2. 2023Exactly conservative physics-informed neural networks and deep operator networks for dynamical systemscited by 13
  3. 2025Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunitiescited by 11
  4. 2024Learning a Neural Solver for Parametric PDE to Enhance Physics-Informed Methodscited by 10
  5. 2023Improving physics-informed DeepONets with hard constraintscited by 9
  6. 2026Out-of-Distribution Generalization for Neural Physics Solverscited by 5
  7. 2023M-ENIAC: A machine learning recreation of the first successful numerical weather forecastscited by 3
  8. 2023A Sequential Meta-Transfer (SMT) Learning to Combat Complexities of Physics-Informed Neural Networks: Application to Composites Autoclave Processingcited by 3
  9. 2025Gabor-Enhanced Physics-Informed Neural Networks for Fast Simulations of Acoustic Wavefieldscited by 3
  10. 2026TINNs: Time-Induced Neural Networks for Solving Time-Dependent PDEscited by 3
  11. 2025Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equationcited by 2
  12. 2024PinnDE: Physics-Informed Neural Networks for Solving Differential Equationscited by 1
7 more
  1. 2025Solving 2-D Helmholtz equation in the rectangular, circular, and elliptical domains using neural networkscited by 1
  2. 2024Architectural Strategies for the optimization of Physics-Informed Neural Networkscited by 0
  3. 2025Least-Squares-Embedded Optimization for Accelerated Convergence of PINNs in Acoustic Wavefield Simulationscited by 0
  4. 2025Latent Representation Learning of Multi-scale Thermophysics: Application to Dynamics in Shocked Porous Energetic Materialcited by 0
  5. 2025Low-rank adaptive physics-informed HyperDeepONets for solving differential equationscited by 0
  6. 2025Intrinsic-Metric Physics-Informed Neural Networks (IM-PINN) for Reaction-Diffusion Dynamics on Complex Riemannian Manifoldscited by 0
  7. 2026A Comparative Investigation of Thermodynamic Structure-Informed Neural Networkscited by 0