mlsanityData to 7 Oct 2026

Learning Physics-Informed Neural Networks without Stacked Back-propagation

arXiv 2202.09340 ↗First posted 18 Feb 2022cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

Physics-Informed Neural Network (PINN) has become a commonly used machine learning approach to solve partial differential equations (PDE). But, facing high-dimensional secondorder PDE problems, PINN will suffer from severe scalability issues since its loss includes second-order derivatives, the computational cost of which will grow along with the dimension during stacked back-propagation. In this work, we develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. We further discuss the model capacity and provide variance reduction methods to address key limitations in the derivative estimation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is significantly faster. Our code is released at https://github.com/LithiumDA/PINN-without-Stacked-BP.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 18 of its 45 references that are in the map. First posted on arXiv 18 Feb 2022.

Builds on

18 of 45 references.

References · 18

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  5. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  6. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  7. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  8. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  9. 2017PDE-Net: Learning PDEs from Datacited by 304
  10. 2018PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Networkcited by 208
  11. 2020A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learningcited by 160
  12. 2020Randomized Smoothing of All Shapes and Sizescited by 146
6 more
  1. 2020MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radiuscited by 114
  2. 2021CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Methodcited by 93
  3. 2019Physics-Informed Neural Networks for Power Systemscited by 80
  4. 2021Estimating High Order Gradients of the Data Distribution by Denoisingcited by 63
  5. 2020Efficient Learning of Generative Models via Finite-Difference Score Matchingcited by 49
  6. 2020On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEscited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 7; 2025: 6; 2026: 3.

23 papers.

Sort:

  1. 2023Tackling the Curse of Dimensionality with Physics-Informed Neural Networkscited by 78
  2. 2024From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learningcited by 73
  3. 2025CodePDE: An Inference Framework for LLM-driven PDE Solver Generationcited by 37
  4. 2023Hutchinson Trace Estimation for High-Dimensional and High-Order Physics-Informed Neural Networkscited by 23
  5. 2024Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck Equationscited by 22
  6. 2024Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operatorscited by 20
  7. 2023Bias-Variance Trade-off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEscited by 19
  8. 2023BAFFLE: A Baseline of Backpropagation-Free Federated Learningcited by 12
  9. 2023Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networkscited by 12
  10. 2023Forward Laplacian: A New Computational Framework for Neural Network-based Variational Monte Carlocited by 6
  11. 2024DOF: Accelerating High-order Differential Operators with Forward Propagationcited by 5
  12. 2024Separable Operator Networkscited by 4
11 more
  1. 2025Scalable Back-Propagation-Free Training of Optical Physics-Informed Neural Networkscited by 2
  2. 2025Noisy PDE Training Requires Bigger PINNscited by 2
  3. 2023Unsupervised Random Quantum Networks for PDEscited by 1
  4. 2024Score-fPINN: Fractional Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker-Planck-Levy Equationscited by 1
  5. 2024Spatiotemporal Graph Learning with Direct Volumetric Information Passing and Feature Enhancementcited by 1
  6. 2025Maximal Update Parametrization and Zero-Shot Hyperparameter Transfer for Fourier Neural Operatorscited by 1
  7. 2025Tensor-Compressed and Fully-Quantized Training of Neural PDE Solverscited by 1
  8. 2025Mollifier Layers: Enabling Efficient High-Order Derivatives in Inverse PDE Learningcited by 0
  9. 2026Deep Neural networks for solving high-dimensional parabolic partial differential equationscited by 0
  10. 2026Stochastic Dimension Implicit Functional Projections for Global Integral Conservation in High-Dimensional PINNscited by 0
  11. 2026Parameterized Representations via Implicit Stochastic Modulation for High-Dimensional and High-Order Neural PDE Solverscited by 0