# Accelerated Training of Physics-Informed Neural Networks (PINNs) using Meshless Discretizations

arXiv: https://arxiv.org/abs/2205.09332 · HTML: https://mlsanity.com/paper/2205.09332/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 40 references that are in the map.
- First posted on arXiv 19 May 2022.

## Builds on (most cited first, 10 of 10)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems](https://mlsanity.com/paper/2111.02801/) (2021)
- [Efficient Learning of Generative Models via Finite-Difference Score Matching](https://mlsanity.com/paper/2007.03317/) (2020)
- [Robust Node Generation for Meshfree Discretizations on Irregular Domains and Surfaces](https://mlsanity.com/paper/1806.02972/) (2018)
- [Hyperviscosity-Based Stabilization for Radial Basis Function-Finite Difference (RBF-FD) Discretizations of Advection-Diffusion Equations](https://mlsanity.com/paper/1806.03798/) (2018)
- [Finite Difference Neural Networks: Fast Prediction of Partial Differential Equations](https://mlsanity.com/paper/2006.01892/) (2020)
- [A Robust Hyperviscosity Formulation for Stable RBF-FD Discretizations of Advection-Diffusion-Reaction Equations on Manifolds](https://mlsanity.com/paper/1910.07059/) (2019)

## Built on by (most cited first, 27 of 27)

- [Separable Physics-Informed Neural Networks](https://mlsanity.com/paper/2306.15969/) (2023)
- [Unveiling the optimization process of Physics Informed Neural Networks: How accurate and competitive can PINNs be?](https://mlsanity.com/paper/2405.04230/) (2024)
- [Efficient physics-informed neural networks using hash encoding](https://mlsanity.com/paper/2302.13397/) (2023)
- [Neural Spectral Methods: Self-supervised learning in the spectral domain](https://mlsanity.com/paper/2312.05225/) (2023)
- [Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology](https://mlsanity.com/paper/2310.05227/) (2023)
- [Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learning](https://mlsanity.com/paper/2407.00809/) (2024)
- [On the Generalization of PINNs outside the training domain and the Hyperparameters influencing it](https://mlsanity.com/paper/2302.07557/) (2023)
- [Physics-Regularized Multi-Modal Image Assimilation for Brain Tumor Localization](https://mlsanity.com/paper/2409.20409/) (2024)
- [Spectral Informed Neural Network: An Efficient and Low-Memory PINN](https://mlsanity.com/paper/2408.16414/) (2024)
- [Learning PDE Solvers with Physics and Data: A Unifying View of Physics-Informed Neural Networks and Neural Operators](https://mlsanity.com/paper/2601.14517/) (2026)
- [ParticleGS: Learning Neural Gaussian Particle Dynamics from Videos for Prior-free Physical Motion Extrapolation](https://mlsanity.com/paper/2505.20270/) (2025)
- [EPINN-NSE: Enhanced Physics-Informed Neural Networks for Solving Navier-Stokes Equations](https://mlsanity.com/paper/2304.03689/) (2023)
- [An efficient wavelet-based physics-informed neural network for multiscale problems](https://mlsanity.com/paper/2409.11847/) (2024)
- [A Quasilinear Algorithm for Computing Higher-Order Derivatives of Deep Feed-Forward Neural Networks](https://mlsanity.com/paper/2412.09752/) (2024)
- [Harnessing physics-informed operators for high-dimensional reliability analysis problems](https://mlsanity.com/paper/2409.04708/) (2024)
- [FlashPDE: A Drop-In Fused Triton Operator Library for Neural PDE Solvers](https://mlsanity.com/paper/2607.18020/) (2026)
- [HNS: An Efficient Hermite Neural Solver for Solving Time-Fractional Partial Differential Equations](https://mlsanity.com/paper/2310.04789/) (2023)
- [Zero Coordinate Shift: Whetted Automatic Differentiation for Physics-informed Operator Learning](https://mlsanity.com/paper/2311.00860/) (2023)
- [Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUs](https://mlsanity.com/paper/2403.17607/) (2024)
- [Feynman-Kac Operator Expectation Estimator](https://mlsanity.com/paper/2407.02010/) (2024)
- [Regime-Aware Time Weighting for Physics-Informed Neural Networks](https://mlsanity.com/paper/2407.21642/) (2024)
- [Operator Learning for Consolidation: An Architectural Comparison for DeepONet Variants](https://mlsanity.com/paper/2507.10368/) (2025)
- [Forecasting India's Demographic Transition Under Fertility Policy Scenarios Using hybrid LSTM-PINN Model](https://mlsanity.com/paper/2512.00760/) (2025)
- [Derivative Computation in PINNs: Automatic Differentiation, Finite Differences and Beyond](https://mlsanity.com/paper/2608.11020/) (2026)
- [A Computational Comparison of Fourier Spectral Differentiation and Spatial Automatic Differentiation in Periodic Physics-Informed Neural Networks](https://mlsanity.com/paper/2609.02110/) (2026)
- [The limits of exactness: On the failure of automatic differentiation in physics-informed machine learning](https://mlsanity.com/paper/2609.33078/) (2026)
- [Localized Operator Learning with Adaptive Partition-of-Unity Mixture-of-Expert Networks](https://mlsanity.com/paper/2610.04708/) (2026)

## Abstract (arXiv)

We present a new technique for the accelerated training of physics-informed neural networks (PINNs): discretely-trained PINNs (DT-PINNs). The repeated computation of partial derivative terms in the PINN loss functions via automatic differentiation during training is known to be computationally expensive, especially for higher-order derivatives. DT-PINNs are trained by replacing these exact spatial derivatives with high-order accurate numerical discretizations computed using meshless radial basis function-finite differences (RBF-FD) and applied via sparse-matrix vector multiplication. The use of RBF-FD allows for DT-PINNs to be trained even on point cloud samples placed on irregular domain geometries. Additionally, though traditional PINNs (vanilla-PINNs) are typically stored and trained in 32-bit floating-point (fp32) on the GPU, we show that for DT-PINNs, using fp64 on the GPU leads to significantly faster training times than fp32 vanilla-PINNs with comparable accuracy. We demonstrate the efficiency and accuracy of DT-PINNs via a series of experiments. First, we explore the effect of network depth on both numerical and automatic differentiation of a neural network with random weights and show that RBF-FD approximations of third-order accuracy and above are more efficient while being sufficiently accurate. We then compare the DT-PINNs to vanilla-PINNs on both linear and nonlinear Poisson equations and show that DT-PINNs achieve similar losses with 2-4x faster training times on a consumer GPU. Finally, we also demonstrate that similar results can be obtained for the PINN solution to the heat equation (a space-time problem) by discretizing the spatial derivatives using RBF-FD and using automatic differentiation for the temporal derivative. Our results show that fp64 DT-PINNs offer a superior cost-accuracy profile to fp32 vanilla-PINNs.