mlsanityData to 7 Oct 2026

Efficient PINNs via Multi-Head Unimodular Regularization of the Solutions Space

arXiv 2501.12116 ↗First posted 21 Jan 2025cs.LG cs.AI hep-thcited by 6

Not on the map. Why?

Abstract (arXiv)

Non-linear differential equations are a fundamental tool to describe different phenomena in nature. However, we still lack a well-established method to tackle stiff differential equations. Here we present a machine learning framework to facilitate the solution of nonlinear multiscale differential equations and, especially, inverse problems using Physics-Informed Neural Networks (PINNs). This framework is based on what is called textitmulti-head (MH) training, which involves training the network to learn a general space of all solutions for a given set of equations with certain variability, rather than learning a specific solution of the system. This setup is used with a second novel technique that we call Unimodular Regularization (UR) of the latent space of solutions. We show that the multi-head approach, combined with Unimodular Regularization, significantly improves the efficiency of PINNs by facilitating the transfer learning process thereby enabling the finding of solutions for nonlinear, coupled, and multiscale differential equations.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 21 of its 48 references that are in the map. First posted on arXiv 21 Jan 2025.

Builds on

21 of 48 references.

References · 21

  1. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  2. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  3. 2019Hamiltonian Neural Networkscited by 702
  4. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  5. 2019DeepXDE: A deep learning library for solving differential equationscited by 617
  6. 2019Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Datacited by 261
  7. 2019Robust Learning with Jacobian Regularizationcited by 127
  8. 2021One-Shot Transfer Learning of Physics-Informed Neural Networkscited by 49
  9. 2020Hamiltonian neural networks for solving equations of motioncited by 33
  10. 2023L-HYDRA: Multi-Head Physics-Informed Neural Networkscited by 29
  11. 2022Delta-PINNs: physics-informed neural networks on complex geometriescited by 28
  12. 2020Solving Differential Equations Using Neural Network Solution Bundlescited by 20
9 more
  1. 2020Forecasting Hamiltonian dynamics without canonical coordinatescited by 17
  2. 2024DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashingcited by 15
  3. 2023Exactly conservative physics-informed neural networks and deep operator networks for dynamical systemscited by 13
  4. 2022Transfer Learning with Physics-Informed Neural Networks for Efficient Simulation of Branched Flowscited by 12
  5. 2024Gravitational Duals from Equations of Statecited by 10
  6. 2024Data-Guided Physics-Informed Neural Networks for Solving Inverse Problems in Partial Differential Equationscited by 5
  7. 2020RicciNets: Curvature-guided Pruning of High-performance Neural Networks Using Ricci Flowcited by 4
  8. 2023One-Shot Transfer Learning for Nonlinear ODEscited by 3
  9. 2021Thoughts on the Consistency between Ricci Flow and Neural Network Behaviorcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Learning embeddings of non-linear PDEs: the Burgers' equationcited by 0
  2. 2026Recovering Sharp Conductivity Features in the Finite-Data Calder'on Problem with Physics-Informed Neural Networkscited by 0
  3. 2026Gravitational Duals from Equations of State II: Large Hierarchies and False Vacuacited by 0
  4. 2026Physics-Informed Neural Embeddings of PDE Solution Familiescited by 0
  5. 2026Variational Boosting for Physics-Informed Neural Networkscited by 0
  6. 2026Physics-Informed Neural Networks to Infer the Perpendicular Energy Conductivity in the Scrape-Off Layer of Stellarator Devicescited by 0