# Loss Landscape Engineering via Data Regulation on PINNs

arXiv: https://arxiv.org/abs/2205.07843 · HTML: https://mlsanity.com/paper/2205.07843/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 16 May 2022.

## Builds on (most cited first, 9 of 9)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2003.06097/) (2020)
- [PhaseMax: Convex Phase Retrieval via Basis Pursuit](https://mlsanity.com/paper/1610.07531/) (2016)

## Built on by (most cited first, 11 of 11)

- [Data-Driven Physics-Informed Neural Networks: A Digital Twin Perspective](https://mlsanity.com/paper/2401.08667/) (2024)
- [Calibrated Physics-Informed Uncertainty Quantification](https://mlsanity.com/paper/2502.04406/) (2025)
- [Guaranteed prediction sets for functional surrogate models](https://mlsanity.com/paper/2501.18426/) (2025)
- [Neuroevolution of Physics-Informed Neural Nets: Benchmark Problems and Comparative Results](https://mlsanity.com/paper/2212.07624/) (2022)
- [Physics-informed neural networks modeling for systems with moving immersed boundaries: application to an unsteady flow past a plunging foil](https://mlsanity.com/paper/2306.13395/) (2023)
- [Physics-guided training of GAN to improve accuracy in airfoil design synthesis](https://mlsanity.com/paper/2308.10038/) (2023)
- [PTPI-DL-ROMs: pre-trained physics-informed deep learning-based reduced order models for nonlinear parametrized PDEs](https://mlsanity.com/paper/2405.08558/) (2024)
- [ReBaNO: Reduced Basis Neural Operator Mitigating Generalization Gaps and Achieving Discretization Invariance](https://mlsanity.com/paper/2509.09611/) (2025)
- [Unsupervised Random Quantum Networks for PDEs](https://mlsanity.com/paper/2312.14975/) (2023)
- [Visualizing the loss landscapes of physics-informed neural networks](https://mlsanity.com/paper/2602.05849/) (2026)
- [Convergence of physics-informed neural networks modeling time-harmonic wave fields](https://mlsanity.com/paper/2506.11395/) (2025)

## Abstract (arXiv)

Physics-Informed Neural Networks have shown unique utility in parameterising the solution of a well-defined partial differential equation using automatic differentiation and residual losses. Though they provide theoretical guarantees of convergence, in practice the required training regimes tend to be exacting and demanding. Through the course of this paper, we take a deep dive into understanding the loss landscapes associated with a PINN and how that offers some insight as to why PINNs are fundamentally hard to optimise for. We demonstrate how PINNs can be forced to converge better towards the solution, by way of feeding in sparse or coarse data as a regulator. The data regulates and morphs the topology of the loss landscape associated with the PINN to make it easily traversable for the minimiser. Data regulation of PINNs helps ease the optimisation required for convergence by invoking a hybrid unsupervised-supervised training approach, where the labelled data pushes the network towards the vicinity of the solution, and the unlabelled regime fine-tunes it to the solution.