# Efficient PINNs via Multi-Head Unimodular Regularization of the Solutions Space

arXiv: https://arxiv.org/abs/2501.12116 · HTML: https://mlsanity.com/paper/2501.12116/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 48 references that are in the map.
- First posted on arXiv 21 Jan 2025.

## Builds on (most cited first, 21 of 21)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [Robust Learning with Jacobian Regularization](https://mlsanity.com/paper/1908.02729/) (2019)
- [One-Shot Transfer Learning of Physics-Informed Neural Networks](https://mlsanity.com/paper/2110.11286/) (2021)
- [Hamiltonian neural networks for solving equations of motion](https://mlsanity.com/paper/2001.11107/) (2020)
- [L-HYDRA: Multi-Head Physics-Informed Neural Networks](https://mlsanity.com/paper/2301.02152/) (2023)
- [Delta-PINNs: physics-informed neural networks on complex geometries](https://mlsanity.com/paper/2209.03984/) (2022)
- [Solving Differential Equations Using Neural Network Solution Bundles](https://mlsanity.com/paper/2006.14372/) (2020)
- [Forecasting Hamiltonian dynamics without canonical coordinates](https://mlsanity.com/paper/2010.15201/) (2020)
- [DeepRicci: Self-supervised Graph Structure-Feature Co-Refinement for Alleviating Over-squashing](https://mlsanity.com/paper/2401.12780/) (2024)
- [Exactly conservative physics-informed neural networks and deep operator networks for dynamical systems](https://mlsanity.com/paper/2311.14131/) (2023)
- [Transfer Learning with Physics-Informed Neural Networks for Efficient Simulation of Branched Flows](https://mlsanity.com/paper/2211.00214/) (2022)
- [Gravitational Duals from Equations of State](https://mlsanity.com/paper/2403.14763/) (2024)
- [Data-Guided Physics-Informed Neural Networks for Solving Inverse Problems in Partial Differential Equations](https://mlsanity.com/paper/2407.10836/) (2024)
- [RicciNets: Curvature-guided Pruning of High-performance Neural Networks Using Ricci Flow](https://mlsanity.com/paper/2007.04216/) (2020)
- [One-Shot Transfer Learning for Nonlinear ODEs](https://mlsanity.com/paper/2311.14931/) (2023)
- [Thoughts on the Consistency between Ricci Flow and Neural Network Behavior](https://mlsanity.com/paper/2111.08410/) (2021)

## Built on by (most cited first, 6 of 6)

- [Learning embeddings of non-linear PDEs: the Burgers' equation](https://mlsanity.com/paper/2603.07812/) (2026)
- [Recovering Sharp Conductivity Features in the Finite-Data Calder'on Problem with Physics-Informed Neural Networks](https://mlsanity.com/paper/2606.28158/) (2026)
- [Gravitational Duals from Equations of State II: Large Hierarchies and False Vacua](https://mlsanity.com/paper/2606.30117/) (2026)
- [Physics-Informed Neural Embeddings of PDE Solution Families](https://mlsanity.com/paper/2607.06348/) (2026)
- [Variational Boosting for Physics-Informed Neural Networks](https://mlsanity.com/paper/2607.23940/) (2026)
- [Physics-Informed Neural Networks to Infer the Perpendicular Energy Conductivity in the Scrape-Off Layer of Stellarator Devices](https://mlsanity.com/paper/2609.11628/) (2026)

## Abstract (arXiv)

Non-linear differential equations are a fundamental tool to describe different phenomena in nature. However, we still lack a well-established method to tackle stiff differential equations. Here we present a machine learning framework to facilitate the solution of nonlinear multiscale differential equations and, especially, inverse problems using Physics-Informed Neural Networks (PINNs). This framework is based on what is called textitmulti-head (MH) training, which involves training the network to learn a general space of all solutions for a given set of equations with certain variability, rather than learning a specific solution of the system. This setup is used with a second novel technique that we call Unimodular Regularization (UR) of the latent space of solutions. We show that the multi-head approach, combined with Unimodular Regularization, significantly improves the efficiency of PINNs by facilitating the transfer learning process thereby enabling the finding of solutions for nonlinear, coupled, and multiscale differential equations.