# A practical PINN framework for multi-scale problems with multi-magnitude loss terms

arXiv: https://arxiv.org/abs/2308.06672 · HTML: https://mlsanity.com/paper/2308.06672/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 13 Aug 2023.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks](https://mlsanity.com/paper/2012.10047/) (2020)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations](https://mlsanity.com/paper/2107.07871/) (2021)
- [Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains](https://mlsanity.com/paper/2007.11207/) (2020)
- [A Dual-Dimer Method for Training Physics-Constrained Neural Networks with Minimax Architecture](https://mlsanity.com/paper/2005.00615/) (2020)
- [Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios](https://mlsanity.com/paper/2205.07731/) (2022)
- [Subspace Decomposition based DNN algorithm for elliptic type multi-scale PDEs](https://mlsanity.com/paper/2112.06660/) (2021)

## Built on by (most cited first, 15 of 15)

- [Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?](https://mlsanity.com/paper/2405.04230/) (2024)
- [Learnable Activation Functions in Physics-Informed Neural Networks for Solving Partial Differential Equations](https://mlsanity.com/paper/2411.15111/) (2024)
- [Constrained or Unconstrained? Neural-Network-Based Equation Discovery from Data](https://mlsanity.com/paper/2406.02581/) (2024)
- [Enhanced BPINN Training Convergence in Solving General and Multi-scale Elliptic PDEs with Noise](https://mlsanity.com/paper/2408.09340/) (2024)
- [An efficient wavelet-based physics-informed neural network for multiscale problems](https://mlsanity.com/paper/2409.11847/) (2024)
- [Rethinking Input Domains in Physics-Informed Neural Networks via Geometric Compactification Mappings](https://mlsanity.com/paper/2602.16193/) (2026)
- [Physics-informed Deep Learning to Solve Three-dimensional Terzaghi Consolidation Equation: Forward and Inverse Problems](https://mlsanity.com/paper/2401.05439/) (2024)
- [Improved physics-informed neural network in mitigating gradient related failures](https://mlsanity.com/paper/2407.19421/) (2024)
- [DiffGrad for Physics-Informed Neural Networks](https://mlsanity.com/paper/2409.03239/) (2024)
- [Hard constraint learning approaches with trainable influence functions for evolutionary equations](https://mlsanity.com/paper/2502.17497/) (2025)
- [Physics-based machine learning for mantle convection simulations](https://mlsanity.com/paper/2505.16041/) (2025)
- [Wavelet-Accelerated Physics-Informed Quantum Neural Network for Multiscale Partial Differential Equations](https://mlsanity.com/paper/2512.08256/) (2025)
- [More Consistent Accuracy PINN via Alternating Easy-Hard Training](https://mlsanity.com/paper/2512.17607/) (2025)
- [MCMC Informed Neural Emulators for Uncertainty Quantification in Dynamical Systems](https://mlsanity.com/paper/2603.10987/) (2026)
- [An adaptive wavelet-based PINN for problems with localized high-magnitude source](https://mlsanity.com/paper/2604.28180/) (2026)

## Abstract (arXiv)

For multi-scale problems, the conventional physics-informed neural networks (PINNs) face some challenges in obtaining available predictions. In this paper, based on PINNs, we propose a practical deep learning framework for multi-scale problems by reconstructing the loss function and associating it with special neural network architectures. New PINN methods derived from the improved PINN framework differ from the conventional PINN method mainly in two aspects. First, the new methods use a novel loss function by modifying the standard loss function through a (grouping) regularization strategy. The regularization strategy implements a different power operation on each loss term so that all loss terms composing the loss function are of approximately the same order of magnitude, which makes all loss terms be optimized synchronously during the optimization process. Second, for the multi-frequency or high-frequency problems, in addition to using the modified loss function, new methods upgrade the neural network architecture from the common fully-connected neural network to special network architectures such as the Fourier feature architecture, and the integrated architecture developed by us. The combination of the above two techniques leads to a significant improvement in the computational accuracy of multi-scale problems. Several challenging numerical examples demonstrate the effectiveness of the proposed methods. The proposed methods not only significantly outperform the conventional PINN method in terms of computational efficiency and computational accuracy, but also compare favorably with the state-of-the-art methods in the recent literature. The improved PINN framework facilitates better application of PINNs to multi-scale problems.