mlsanityData to 7 Oct 2026

Hypernetwork-based Meta-Learning for Low-Rank Physics-Informed Neural Networks

arXiv 2310.09528 ↗First posted 14 Oct 2023cs.LG cs.NA math.NAcited by 42

Not on the map. Why?

Abstract (arXiv)

In various engineering and applied science applications, repetitive numerical simulations of partial differential equations (PDEs) for varying input parameters are often required (e.g., aircraft shape optimization over many design parameters) and solvers are required to perform rapid execution. In this study, we suggest a path that potentially opens up a possibility for physics-informed neural networks (PINNs), emerging deep-learning-based solvers, to be considered as one such solver. Although PINNs have pioneered a proper integration of deep-learning and scientific computing, they require repetitive time-consuming training of neural networks, which is not suitable for many-query scenarios. To address this issue, we propose a lightweight low-rank PINNs containing only hundreds of model parameters and an associated hypernetwork-based meta-learning algorithm, which allows efficient approximation of solutions of PDEs for varying ranges of PDE input parameters. Moreover, we show that the proposed method is effective in overcoming a challenging issue, known as "failure modes" of PINNs.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 21 of its 47 references that are in the map. First posted on arXiv 14 Oct 2023.

Builds on

21 of 47 references.

References · 21

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  4. 2018On the Spectral Bias of Neural Networkscited by 1,321
  5. 2016HyperNetworkscited by 1,003
  6. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  7. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  8. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  9. 2020B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Datacited by 329
  10. 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 283
  11. 2020Learned Initializations for Optimizing Coordinate-Based Neural Representationscited by 240
  12. 2020Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanismcited by 231
9 more
  1. 2021The Low-Rank Simplicity Bias in Deep Networkscited by 150
  2. 2020The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networkscited by 58
  3. 2021Initialization and Regularization of Factorized Neural Layerscited by 58
  4. 2020Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsificationcited by 52
  5. 2021Meta-Learning Sparse Implicit Neural Representationscited by 48
  6. 2020DPM: A Novel Training Method for Physics-Informed Neural Networks in Extrapolationcited by 45
  7. 2021HyperPINN: Learning parameterized differential equations with physics-informed hypernetworkscited by 38
  8. 2022Meta-Learning Sparse Compression Networkscited by 22
  9. 2022SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decompositioncited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 11; 2025: 18; 2026: 11.

42 papers.

Sort:

  1. 2024Parameterized Physics-informed Neural Networks for Parameterized PDEscited by 35
  2. 2024Dual Cone Gradient Descent for Training Physics-Informed Neural Networkscited by 21
  3. 2023Learning Flexible Body Collision Dynamics with Hierarchical Contact Mesh Transformercited by 18
  4. 2025Transfer Learning in Physics-Informed Neural Networks: Full Fine-Tuning, Lightweight Fine-Tuning, and Low-Rank Adaptationcited by 14
  5. 2024CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equationscited by 12
  6. 2025Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunitiescited by 10
  7. 2024Physics-Regularized Multi-Modal Image Assimilation for Brain Tumor Localizationcited by 9
  8. 2024OmniArch: Building Foundation Model For Scientific Computingcited by 8
  9. 2025HyPINO: Multi-Physics Neural Operators via HyperPINNs and the Method of Manufactured Solutionscited by 8
  10. 2024GEPS: Boosting Generalization in Parametric PDE Neural Solvers through Adaptive Conditioningcited by 7
  11. 2025An Overview of Low-Rank Structures in the Training and Adaptation of Large Modelscited by 7
  12. 2023Evolutionary Optimization of Physics-Informed Neural Networks: Advancing Generalizability by the Baldwin Effectcited by 5
30 more
  1. 2025Adaptive Physics-informed Neural Networks: A Surveycited by 5
  2. 2025Towards a Physics Foundation Modelcited by 5
  3. 2024A Low Rank Neural Representation of Entropy Solutionscited by 3
  4. 2024FastLRNR and Sparse Physics Informed Backpropagationcited by 3
  5. 2025Residuals Are Not Enough: Limits of Physics-Informed Pre-Training for Scientific Foundation Modelscited by 3
  6. 2025F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learningcited by 3
  7. 2025Fourier-Modulated Implicit Neural Representation for Multispectral Satellite Image Compressioncited by 2
  8. 2025A Two-Phase Deep Learning Framework for Adaptive Time-Stepping in High-Speed Flow Modelingcited by 2
  9. 2025Neural Functions for Learning Periodic Signalcited by 2
  10. 2025Meta-learning Structure-Preserving Dynamicscited by 2
  11. 2025Conditional neural field for spatial dimension reduction of turbulence data: a comparison studycited by 2
  12. 2024Neural Parameter Regression for Explicit Representations of PDE Solution Operatorscited by 1
  13. 2025Unveiling the Potential of Superexpressive Networks in Implicit Neural Representationscited by 1
  14. 2025Enhancing Stability of Physics-Informed Neural Network Training Through Saddle-Point Reformulationcited by 1
  15. 2026Nonlinear model reduction for transport-dominated problemscited by 1
  16. 2026Compositional Meta-Learning for Mitigating Task Heterogeneity in Physics-Informed Neural Networkscited by 1
  17. 2026Extending Fourier Neural Operators for Modeling Parameterized and Coupled PDEscited by 1
  18. 2024Scalable physical source-to-field inference with hypernetworkscited by 0
  19. 2024MaD-Scientist: AI-based Scientist solving Convection-Diffusion-Reaction Equations Using Massive PINN-Based Prior Datacited by 0
  20. 2025Beyond Static Models: Hypernetworks for Adaptive and Generalizable Forecasting in Complex Parametric Dynamical Systemscited by 0
  21. 2025Learning Low Rank Neural Representations of Hyperbolic Wave Dynamics from Datacited by 0
  22. 2025Basis-Oriented Low-rank Transfer for Few-Shot and Test-Time Adaptationcited by 0
  23. 2026ODELoRA: Training Low-Rank Adaptation by Solving Ordinary Differential Equationscited by 0
  24. 2026Transferable Physics-Informed Representations via Closed-Form Head Adaptationcited by 0
  25. 2026LIGO-PINN: Learned Initialization via Gated Optimization to Alleviate Convergence Failures in Physics Informed Neural Networkscited by 0
  26. 2026FLARE++: Low-rank attention with dynamic attention routingcited by 0
  27. 2026Physics-Informed Foresight Pruning for Sparse PINN Solvers of Nonlinear PDEscited by 0
  28. 2026Amortizing Physics-Informed Neural Solvers via Graph Hypernetworkscited by 0
  29. 2026Does Text Steer Neural PDE Surrogates? A Controlled Diagnostic with OperatorCLIPcited by 0
  30. 2026Beyond Simulation: Retain-and-Repair Neural Operators for Real-World Adaptationcited by 0