# DOF: Accelerating High-order Differential Operators with Forward Propagation

arXiv: https://arxiv.org/abs/2402.09730 · HTML: https://mlsanity.com/paper/2402.09730/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 15 Feb 2024.

## Builds on (most cited first, 9 of 9)

- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Solving high-dimensional partial differential equations using deep learning](https://mlsanity.com/paper/1707.02568/) (2017)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method](https://mlsanity.com/paper/2110.15832/) (2021)
- [Separable Physics-Informed Neural Networks](https://mlsanity.com/paper/2306.15969/) (2023)
- [Estimating High Order Gradients of the Data Distribution by Denoising](https://mlsanity.com/paper/2111.04726/) (2021)
- [Efficient Learning of Generative Models via Finite-Difference Score Matching](https://mlsanity.com/paper/2007.03317/) (2020)
- [Learning Physics-Informed Neural Networks without Stacked Back-propagation](https://mlsanity.com/paper/2202.09340/) (2022)
- [Hutchinson Trace Estimation for High-Dimensional and High-Order Physics-Informed Neural Networks](https://mlsanity.com/paper/2312.14499/) (2023)

## Built on by (most cited first, 5 of 5)

- [Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks](https://mlsanity.com/paper/2405.15603/) (2024)
- [Stochastic Taylor Derivative Estimator: Efficient amortization for arbitrary differential operators](https://mlsanity.com/paper/2412.00088/) (2024)
- [Collapsing Taylor Mode Automatic Differentiation](https://mlsanity.com/paper/2505.13644/) (2025)
- [Deep Neural networks for solving high-dimensional parabolic partial differential equations](https://mlsanity.com/paper/2601.13256/) (2026)
- [Stochastic Dimension Zeroth-Order Estimator: Stable and Memory-Efficient Training of PINNs](https://mlsanity.com/paper/2603.24002/) (2026)

## Abstract (arXiv)

Solving partial differential equations (PDEs) efficiently is essential for analyzing complex physical systems. Recent advancements in leveraging deep learning for solving PDE have shown significant promise. However, machine learning methods, such as Physics-Informed Neural Networks (PINN), face challenges in handling high-order derivatives of neural network-parameterized functions. Inspired by Forward Laplacian, a recent method of accelerating Laplacian computation, we propose an efficient computational framework, Differential Operator with Forward-propagation (DOF), for calculating general second-order differential operators without losing any precision. We provide rigorous proof of the advantages of our method over existing methods, demonstrating two times improvement in efficiency and reduced memory consumption on any architectures. Empirical results illustrate that our method surpasses traditional automatic differentiation (AutoDiff) techniques, achieving 2x improvement on the MLP structure and nearly 20x improvement on the MLP with Jacobian sparsity.