# DiffGrad for Physics-Informed Neural Networks

arXiv: https://arxiv.org/abs/2409.03239 · HTML: https://mlsanity.com/paper/2409.03239/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 43 references that are in the map.
- First posted on arXiv 5 Sep 2024.

## Builds on (most cited first, 13 of 13)

- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1801.06637/) (2018)
- [Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations](https://mlsanity.com/paper/1811.02033/) (2018)
- [CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method](https://mlsanity.com/paper/2110.15832/) (2021)
- [Physics-informed neural network for ultrasound nondestructive quantification of surface breaking cracks](https://mlsanity.com/paper/2005.03596/) (2020)
- [Physics-informed deep learning for incompressible laminar flows](https://mlsanity.com/paper/2002.10558/) (2020)
- [A practical PINN framework for multi-scale problems with multi-magnitude loss terms](https://mlsanity.com/paper/2308.06672/) (2023)
- [Neural oscillators for generalization of physics-informed machine learning](https://mlsanity.com/paper/2308.08989/) (2023)
- [Augmenting a Physics-Informed Neural Network for the 2D Burgers Equation by Addition of Solution Data Points](https://mlsanity.com/paper/2301.07824/) (2023)
- [Amplifying Sine Unit: An Oscillatory Activation Function for Deep Neural Networks to Recover Nonlinear Oscillations Efficiently](https://mlsanity.com/paper/2304.09759/) (2023)

## Abstract (arXiv)

Physics-Informed Neural Networks (PINNs) are regarded as state-of-the-art tools for addressing highly nonlinear problems based on partial differential equations. Despite their broad range of applications, PINNs encounter several performance challenges, including issues related to efficiency, minimization of computational cost, and enhancement of accuracy. Burgers' equation, a fundamental equation in fluid dynamics that is extensively used in PINNs, provides flexible results with the Adam optimizer that does not account for past gradients. This paper introduces a novel strategy for solving Burgers' equation by incorporating DiffGrad with PINNs, a method that leverages the difference between current and immediately preceding gradients to enhance performance. A comprehensive computational analysis is conducted using optimizers such as Adam, Adamax, RMSprop, and DiffGrad to evaluate and compare their effectiveness. Our approach includes visualizing the solutions over space at various time intervals to demonstrate the accuracy of the network. The results show that DiffGrad not only improves the accuracy of the solution but also reduces training time compared to the other optimizers.