# Deep learning approaches to surrogates for solving the diffusion equation for mechanistic real-world simulations

arXiv: https://arxiv.org/abs/2102.05527 · HTML: https://mlsanity.com/paper/2102.05527/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 47 references that are in the map.
- First posted on arXiv 10 Feb 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Study](https://mlsanity.com/paper/2004.03271/) (2020)
- [Machine learning for molecular simulation](https://mlsanity.com/paper/1911.02792/) (2019)
- [Fashionable Modelling with Flux](https://mlsanity.com/paper/1811.01457/) (2018)
- [DiffEqFlux.jl - A Julia Library for Neural Differential Equations](https://mlsanity.com/paper/1902.02376/) (2019)
- [Deep Learning the Physics of Transport Phenomena](https://mlsanity.com/paper/1709.02432/) (2017)
- [Molecular enhanced sampling with autoencoders: On-the-fly collective variable discovery and accelerated free energy landscape exploration](https://mlsanity.com/paper/1801.00203/) (2017)
- [Weakly-Supervised Deep Learning of Heat Transport via Physics Informed Loss](https://mlsanity.com/paper/1807.11374/) (2018)
- [An unsupervised learning approach to solving heat equations on chip based on Auto Encoder and Image Gradient](https://mlsanity.com/paper/2007.09684/) (2020)
- [Learning Everywhere: A Taxonomy for the Integration of Machine Learning and Simulations](https://mlsanity.com/paper/1909.13340/) (2019)
- [Learning Everywhere: Pervasive Machine Learning for Effective High-Performance Computation](https://mlsanity.com/paper/1902.10810/) (2019)

## Built on by (most cited first, 5 of 5)

- [Analyzing the Performance of Deep Encoder-Decoder Networks as Surrogates for a Diffusion Equation](https://mlsanity.com/paper/2302.03786/) (2023)
- [Feasibility Study on Active Learning of Smart Surrogates for Scientific Simulations](https://mlsanity.com/paper/2407.07674/) (2024)
- [Velocity continuation with Fourier neural operators for accelerated uncertainty quantification](https://mlsanity.com/paper/2203.14386/) (2022)
- [Surrogate modeling of Cellular-Potts Agent-Based Models as a segmentation task using the U-Net neural network architecture](https://mlsanity.com/paper/2505.00316/) (2025)
- [Generative diffusion model surrogates for mechanistic agent-based biological models](https://mlsanity.com/paper/2505.09630/) (2025)

## Abstract (arXiv)

In many mechanistic medical, biological, physical and engineered spatiotemporal dynamic models the numerical solution of partial differential equations (PDEs) can make simulations impractically slow. Biological models require the simultaneous calculation of the spatial variation of concentration of dozens of diffusing chemical species. Machine learning surrogates, neural networks trained to provide approximate solutions to such complicated numerical problems, can often provide speed-ups of several orders of magnitude compared to direct calculation. PDE surrogates enable use of larger models than are possible with direct calculation and can make including such simulations in real-time or near-real time workflows practical. Creating a surrogate requires running the direct calculation tens of thousands of times to generate training data and then training the neural network, both of which are computationally expensive. We use a Convolutional Neural Network to approximate the stationary solution to the diffusion equation in the case of two equal-diameter, circular, constant-value sources located at random positions in a two-dimensional square domain with absorbing boundary conditions. To improve convergence during training, we apply a training approach that uses roll-back to reject stochastic changes to the network that increase the loss function. The trained neural network approximation is about 1e3 times faster than the direct calculation for individual replicas. Because different applications will have different criteria for acceptable approximation accuracy, we discuss a variety of loss functions and accuracy estimators that can help select the best network for a particular application.