# dNNsolve: an efficient NN-based PDE solver

arXiv: https://arxiv.org/abs/2103.08662 · HTML: https://mlsanity.com/paper/2103.08662/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 63 references that are in the map.
- First posted on arXiv 15 Mar 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations](https://mlsanity.com/paper/1706.04702/) (2017)
- [Training behavior of deep neural network in frequency domain](https://mlsanity.com/paper/1807.01251/) (2018)
- [Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10566/) (2017)
- [Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flows](https://mlsanity.com/paper/1810.08217/) (2018)
- [Accelerating Eulerian Fluid Simulation With Convolutional Networks](https://mlsanity.com/paper/1607.03597/) (2016)
- [Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks](https://mlsanity.com/paper/1906.05747/) (2019)
- [Adversarial Uncertainty Quantification in Physics-Informed Neural Networks](https://mlsanity.com/paper/1811.04026/) (2018)
- [An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications](https://mlsanity.com/paper/1908.10407/) (2019)
- [Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations](https://mlsanity.com/paper/1709.05963/) (2017)
- [Deep Relaxation: partial differential equations for optimizing deep neural networks](https://mlsanity.com/paper/1704.04932/) (2017)
- [PyDEns: a Python Framework for Solving Differential Equations with Neural Networks](https://mlsanity.com/paper/1909.11544/) (2019)
- [A Discussion on Solving Partial Differential Equations using Neural Networks](https://mlsanity.com/paper/1904.07200/) (2019)
- [Locally adaptive activation functions with slope recovery term for deep and physics-informed neural networks](https://mlsanity.com/paper/1909.12228/) (2019)
- [Neural networks catching up with finite differences in solving partial differential equations in higher dimensions](https://mlsanity.com/paper/1712.05067/) (2017)
- [The DUNE Framework: Basic Concepts and Recent Developments](https://mlsanity.com/paper/1909.13672/) (2019)

## Built on by (most cited first, 4 of 4)

- [Elvet -- a neural network-based differential equation and variational problem solver](https://mlsanity.com/paper/2103.14575/) (2021)
- [Study of Drug Assimilation in Human System using Physics Informed Neural Networks](https://mlsanity.com/paper/2110.05531/) (2021)
- [Qade: Solving Differential Equations on Quantum Annealers](https://mlsanity.com/paper/2204.03657/) (2022)
- [Transferability of Winning Lottery Tickets in Neural Network Differential Equation Solvers](https://mlsanity.com/paper/2306.09863/) (2023)

## Abstract (arXiv)

Neural Networks (NNs) can be used to solve Ordinary and Partial Differential Equations (ODEs and PDEs) by redefining the question as an optimization problem. The objective function to be optimized is the sum of the squares of the PDE to be solved and of the initial/boundary conditions. A feed forward NN is trained to minimise this loss function evaluated on a set of collocation points sampled from the domain where the problem is defined. A compact and smooth solution, that only depends on the weights of the trained NN, is then obtained. This approach is often referred to as PINN, from Physics Informed Neural Network~citeraissi2017physics1, raissi2017physics2. Despite the success of the PINN approach in solving various classes of PDEs, an implementation of this idea that is capable of solving a large class of ODEs and PDEs with good accuracy and without the need to finely tune the hyperparameters of the network, is not available yet. In this paper, we introduce a new implementation of this concept - called dNNsolve - that makes use of dual Neural Networks to solve ODEs/PDEs. These include: i) sine and sigmoidal activation functions, that provide a more efficient basis to capture both secular and periodic patterns in the solutions; ii) a newly designed architecture, that makes it easy for the the NN to approximate the solution using the basis functions mentioned above. We show that dNNsolve is capable of solving a broad range of ODEs/PDEs in 1, 2 and 3 spacetime dimensions, without the need of hyperparameter fine-tuning.