mlsanityData to 14 Oct 2026

Loss Landscape Engineering via Data Regulation on PINNs

arXiv 2205.07843 ↗First posted 16 May 2022cs.LG physics.comp-phcited by 11

Not on the map. Why?

Abstract (arXiv)

Physics-Informed Neural Networks have shown unique utility in parameterising the solution of a well-defined partial differential equation using automatic differentiation and residual losses. Though they provide theoretical guarantees of convergence, in practice the required training regimes tend to be exacting and demanding. Through the course of this paper, we take a deep dive into understanding the loss landscapes associated with a PINN and how that offers some insight as to why PINNs are fundamentally hard to optimise for. We demonstrate how PINNs can be forced to converge better towards the solution, by way of feeding in sparse or coarse data as a regulator. The data regulates and morphs the topology of the loss landscape associated with the PINN to make it easily traversable for the minimiser. Data regulation of PINNs helps ease the optimisation required for convergence by invoking a hybrid unsupervised-supervised training approach, where the labelled data pushes the network towards the vicinity of the solution, and the unlabelled regime fine-tunes it to the solution.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 9 of its 23 references that are in the map. First posted on arXiv 16 May 2022.

Builds on

9 of 23 references.

References · 9

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  4. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  5. 2015Automatic differentiation in machine learning: a surveycited by 855
  6. 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
  7. 2019DeepXDE: A deep learning library for solving differential equationscited by 619
  8. 2020B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Datacited by 335
  9. 2016PhaseMax: Convex Phase Retrieval via Basis Pursuitcited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 2; 2025: 4; 2026: 1.

11 papers.

  1. 2024Data-Driven Physics-Informed Neural Networks: A Digital Twin Perspectivecited by 12
  2. 2025Calibrated Physics-Informed Uncertainty Quantificationcited by 8
  3. 2025Guaranteed prediction sets for functional surrogate modelscited by 7
  4. 2022Neuroevolution of Physics-Informed Neural Nets: Benchmark Problems and Comparative Resultscited by 4
  5. 2023Physics-informed neural networks modeling for systems with moving immersed boundaries: application to an unsteady flow past a plunging foilcited by 4
  6. 2023Physics-guided training of GAN to improve accuracy in airfoil design synthesiscited by 4
  7. 2024PTPI-DL-ROMs: pre-trained physics-informed deep learning-based reduced order models for nonlinear parametrized PDEscited by 4
  8. 2025ReBaNO: Reduced Basis Neural Operator Mitigating Generalization Gaps and Achieving Discretization Invariancecited by 2
  9. 2023Unsupervised Random Quantum Networks for PDEscited by 1
  10. 2026Visualizing the loss landscapes of physics-informed neural networkscited by 1
  11. 2025Convergence of physics-informed neural networks modeling time-harmonic wave fieldscited by 0