# Data vs. Physics: The Apparent Pareto Front of Physics-Informed Neural Networks

arXiv: https://arxiv.org/abs/2105.00862 · HTML: https://mlsanity.com/paper/2105.00862/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 43 references that are in the map.
- First posted on arXiv 3 May 2021.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Physics-Informed Neural Networks for Multiphysics Data Assimilation with Application to Subsurface Transport](https://mlsanity.com/paper/1912.02968/) (2019)
- [Accelerating Physics-Informed Neural Network Training with Prior Dictionaries](https://mlsanity.com/paper/2004.08151/) (2020)

## Built on by (most cited first, 27 of 27)

- [Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications](https://mlsanity.com/paper/2211.08064/) (2022)
- [PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEs](https://mlsanity.com/paper/2306.08827/) (2023)
- [Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs)](https://mlsanity.com/paper/2205.01059/) (2022)
- [Mixed formulation of physics-informed neural networks for thermo-mechanically coupled systems and heterogeneous domains](https://mlsanity.com/paper/2302.04954/) (2023)
- [Calibrating constitutive models with full-field data via physics informed neural networks](https://mlsanity.com/paper/2203.16577/) (2022)
- [Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities](https://mlsanity.com/paper/2501.06572/) (2025)
- [Bayesian Physics Informed Neural Networks for Data Assimilation and Spatio-Temporal Modelling of Wildfires](https://mlsanity.com/paper/2212.00970/) (2022)
- [PINN Training using Biobjective Optimization: The Trade-off between Data Loss and Residual Loss](https://mlsanity.com/paper/2302.01810/) (2023)
- [Multi-objective Deep Learning: Taxonomy and Survey of the State of the Art](https://mlsanity.com/paper/2412.01566/) (2024)
- [Error convergence and engineering-guided hyperparameter search of PINNs: towards optimized I-FENN performance](https://mlsanity.com/paper/2303.03918/) (2023)
- [A Conformal Prediction Framework for Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/2509.13717/) (2025)
- [Neuroevolution of Physics-Informed Neural Nets: Benchmark Problems and Comparative Results](https://mlsanity.com/paper/2212.07624/) (2022)
- [PINNverse: Accurate parameter estimation in differential equations from noisy data with constrained physics-informed neural networks](https://mlsanity.com/paper/2504.05248/) (2025)
- [A connection between probability, physics and neural networks](https://mlsanity.com/paper/2209.12737/) (2022)
- [Common pitfalls to avoid while using multiobjective optimization in machine learning](https://mlsanity.com/paper/2405.01480/) (2024)
- [Physics-informed neural network estimation of active material properties in time-dependent cardiac biomechanical models](https://mlsanity.com/paper/2505.03382/) (2025)
- [Multi-material Multi-physics Topology Optimization with Physics-informed Gaussian Process Priors](https://mlsanity.com/paper/2602.17783/) (2026)
- [Scaling Laws of Machine Learning for Optimal Power Flow](https://mlsanity.com/paper/2601.02706/) (2026)
- [On the Role of Consistency Between Physics and Data in Physics-Informed Neural Networks](https://mlsanity.com/paper/2602.10611/) (2026)
- [Bringing Chemistry to Scale: Loss Weight Adjustment for Multivariate Regression in Deep Learning of Thermochemical Processes](https://mlsanity.com/paper/2308.01954/) (2023)
- [Physics-informed Neural Networks for Encoding Dynamics in Real Physical Systems](https://mlsanity.com/paper/2401.03534/) (2024)
- [Reliable and efficient inverse analysis using physics-informed neural networks with normalized distance functions and adaptive weight tuning](https://mlsanity.com/paper/2504.18091/) (2025)
- [B-PL-PINN: Stabilizing PINN Training with Bayesian Pseudo Labeling](https://mlsanity.com/paper/2507.01714/) (2025)
- [Gaussian Process Regression of Steering Vectors With Physics-Aware Deep Composite Kernels for Augmented Listening](https://mlsanity.com/paper/2509.02571/) (2025)
- [Finite basis physics-informed neural networks with hard constraints for viscous fluid flow in highly perforated domains](https://mlsanity.com/paper/2608.08114/) (2026)
- [Beyond Field Accuracy: Two-Axis Diagnosis of Inverse-PINN Parameter Error](https://mlsanity.com/paper/2608.15373/) (2026)
- [Hyperparameter selection for equation learning with biologically-informed neural networks](https://mlsanity.com/paper/2610.02954/) (2026)

## Abstract (arXiv)

Physics-informed neural networks (PINNs) have emerged as a promising deep learning method, capable of solving forward and inverse problems governed by differential equations. Despite their recent advance, it is widely acknowledged that PINNs are difficult to train and often require a careful tuning of loss weights when data and physics loss functions are combined by scalarization of a multi-objective (MO) problem. In this paper, we aim to understand how parameters of the physical system, such as characteristic length and time scales, the computational domain, and coefficients of differential equations affect MO optimization and the optimal choice of loss weights. Through a theoretical examination of where these system parameters appear in PINN training, we find that they effectively and individually scale the loss residuals, causing imbalances in MO optimization with certain choices of system parameters. The immediate effects of this are reflected in the apparent Pareto front, which we define as the set of loss values achievable with gradient-based training and visualize accordingly. We empirically verify that loss weights can be used successfully to compensate for the scaling of system parameters, and enable the selection of an optimal solution on the apparent Pareto front that aligns well with the physically valid solution. We further demonstrate that by altering the system parameterization, the apparent Pareto front can shift and exhibit locally convex parts, resulting in a wider range of loss weights for which gradient-based training becomes successful. This work explains the effects of system parameters on MO optimization in PINNs, and highlights the utility of proposed loss weighting schemes.