mlsanityData to 14 Oct 2026

A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks

arXiv 2207.10289 ↗First posted 21 Jul 2022physics.comp-ph cs.LGcited by 236

Not on the map. Why?

Abstract (arXiv)

Physics-informed neural networks (PINNs) have shown to be an effective tool for solving forward and inverse problems of partial differential equations (PDEs). PINNs embed the PDEs into the loss of the neural network, and this PDE loss is evaluated at a set of scattered residual points. The distribution of these points are highly important to the performance of PINNs. However, in the existing studies on PINNs, only a few simple residual point sampling methods have mainly been used. Here, we present a comprehensive study of two categories of sampling: non-adaptive uniform sampling and adaptive nonuniform sampling. We consider six uniform sampling, including (1) equispaced uniform grid, (2) uniformly random sampling, (3) Latin hypercube sampling, (4) Halton sequence, (5) Hammersley sequence, and (6) Sobol sequence. We also consider a resampling strategy for uniform sampling. To improve the sampling efficiency and the accuracy of PINNs, we propose two new residual-based adaptive sampling methods: residual-based adaptive distribution (RAD) and residual-based adaptive refinement with distribution (RAR-D), which dynamically improve the distribution of residual points based on the PDE residuals during training. Hence, we have considered a total of 10 different sampling methods, including six non-adaptive uniform sampling, uniform sampling with resampling, two proposed adaptive sampling, and an existing adaptive sampling. We extensively tested the performance of these sampling methods for four forward problems and two inverse problems in many setups. Our numerical results presented in this study are summarized from more than 6000 simulations of PINNs. We show that the proposed adaptive sampling methods of RAD and RAR-D significantly improve the accuracy of PINNs with fewer residual points. The results obtained in this study can also be used as a practical guideline in choosing sampling methods.

Key facts. Cited by 236 papers in the map (data through 14 Oct 2026). Builds on 20 of its 51 references that are in the map. First posted on arXiv 21 Jul 2022.

Builds on

20 of 51 references.

References · 20

  1. 2021Characterizing possible failure modes in physics-informed neural networkscited by 643
  2. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 634
  3. 2019DeepXDE: A deep learning library for solving differential equationscited by 619
  4. 2021Physics-informed neural networks with hard constraints for inverse designcited by 280
  5. 2020Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanismcited by 233
  6. 2021Gradient-enhanced physics-informed neural networks for forward and inverse PDE problemscited by 222
  7. 2022Respecting causality is all you need for training physics-informed neural networkscited by 156
  8. 2018Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problemscited by 147
  9. 2020Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networkscited by 123
  10. 2019PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEscited by 118
  11. 2021Parallel Physics-Informed Neural Networks via Domain Decompositioncited by 112
  12. 2021Efficient training of physics-informed neural networks via importance samplingcited by 110
8 more
  1. 2020A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networkscited by 87
  2. 2021Meta-learning PINN loss functionscited by 51
  3. 2022Investigating molecular transport in the human brain from MRI with physics-informed neural networkscited by 24
  4. 2022Systems Biology: Identifiability analysis and parameter identification via systems-biology informed neural networkscited by 23
  5. 2022Improved Training of Physics-Informed Neural Networks with Model Ensemblescited by 19
  6. 2022RANG: A Residual-based Adaptive Node Generation Method for Physics-Informed Neural Networkscited by 14
  7. 2022State-of-the-Art Review of Design of Experiments for Physics-Informed Deep Learningcited by 12
  8. 2022Revisiting PINNs: Generative Adversarial Physics-informed Neural Networks and Point-weighting Methodcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 10; 2023: 38; 2024: 62; 2025: 67; 2026: 59.

236 papers.

Sort:

By area:

  1. 2023An Expert's Guide to Training Physics-informed Neural Networkscited by 126
  2. 2024Challenges in Training PINNs: A Loss Landscape Perspectivecited by 113
  3. 2024A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networkscited by 104
  4. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 92
  5. 2023PINNacle: A Comprehensive Benchmark of Physics-Informed Neural Networks for Solving PDEscited by 87
  6. 2024From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learningcited by 73
  7. 2023Residual-based attention in physics-informed neural networkscited by 72
  8. 2024PirateNets: Physics-informed Deep Learning with Residual Adaptive Networkscited by 72
  9. 2022Failure-informed adaptive sampling for PINNscited by 64
  10. 2025Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspectivecited by 56
  11. 2023Multi-stage Neural Networks: Function Approximator of Machine Precisioncited by 48
  12. 2023Fourier-MIONet: Fourier-enhanced multiple-input neural operators for multiphase modeling of geological carbon sequestrationcited by 46
38 more
  1. 2025Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networkscited by 42
  2. 2024Adaptive Training of Grid-Dependent Physics-Informed Kolmogorov-Arnold Networkscited by 38
  3. 2024Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problemscited by 35
  4. 2024PINNACLE: PINN Adaptive ColLocation and Experimental points selectioncited by 32
  5. 2024Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?cited by 31
  6. 2024RoPINN: Region Optimized Physics-Informed Neural Networkscited by 26
  7. 2023Rethinking materials simulations: Blending direct numerical simulations with neural operatorscited by 25
  8. 2024Active Learning for Neural PDE Solverscited by 22
  9. 2023Efficient physics-informed neural networks using hash encodingcited by 21
  10. 2023Stacked networks improve physics-informed training: applications to neural networks and deep operator networkscited by 21
  11. 2024Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networkscited by 21
  12. 2024Dual Cone Gradient Descent for Training Physics-Informed Neural Networkscited by 21
  13. 2024Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networkscited by 20
  14. 2023Improving physics-informed neural networks with meta-learned optimizationcited by 19
  15. 2023PPDONet: Deep Operator Networks for Fast Prediction of Steady-State Solutions in Disk-Planet Systemscited by 19
  16. 2023Adversarial Adaptive Sampling: Unify PINN and Optimal Transport for the Approximation of PDEscited by 18
  17. 2023Active Learning and Bayesian Optimization: a Unified Perspective to Learn with a Goalcited by 17
  18. 2023An adaptive augmented Lagrangian method for training physics and equality constrained artificial neural networkscited by 16
  19. 2024Separable DeepONet: Breaking the Curse of Dimensionality in Physics-Informed Machine Learningcited by 15
  20. 2023Failure-informed adaptive sampling for PINNs, Part II: combining with re-sampling and subset simulationcited by 13
  21. 2023A multifidelity approach to continual learning for physical systemscited by 13
  22. 2023Fourier neural operator for learning solutions to macroscopic traffic flow models: Application to the forward and inverse problemscited by 13
  23. 2024Learning in PINNs: Phase transition, diffusion equilibrium, and generalizationcited by 13
  24. 2024Data-Driven Physics-Informed Neural Networks: A Digital Twin Perspectivecited by 12
  25. 2025Operator Learning for Reconstructing Flow Fields from Sparse Measurements: an Energy Transformer Approachcited by 12
  26. 2025FP64 is All You Need: Rethinking Failure Modes in Physics-Informed Neural Networkscited by 12
  27. 2024Efficient and generalizable nested Fourier-DeepONet for three-dimensional geological carbon sequestrationcited by 11
  28. 2024PIG: Physics-Informed Gaussians as Adaptive Parametric Mesh Representationscited by 11
  29. 2025Challenges and Advancements in Modeling Shock Fronts with Physics-Informed Neural Networks: A Review and Benchmarking Studycited by 11
  30. 2025Lang-PINN: From Language to Physics-Informed Neural Networks via a Multi-Agent Frameworkcited by 11
  31. 2026Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networkscited by 11
  32. 2023On the Generalization of PINNs outside the training domain and the Hyperparameters influencing itcited by 10
  33. 2023Achieving High Accuracy with PINNs via Energy Natural Gradientscited by 10
  34. 2023Coupling parameter and particle dynamics for adaptive sampling in Neural Galerkin schemescited by 10
  35. 2025Sub-Sequential Physics-Informed Learning with State Space Modelcited by 10
  36. 2024Initialization-enhanced Physics-Informed Neural Network with Domain Decomposition (IDPINN)cited by 9
  37. 2023Feature-adjacent multi-fidelity physics-informed machine learning for partial differential equationscited by 8
  38. 2024Speeding up and reducing memory usage for scientific machine learning via mixed precisioncited by 8

50 most cited of 236 All 236 (JSON)