# Learning effective stochastic differential equations from microscopic simulations: linking stochastic numerics to deep learning

arXiv: https://arxiv.org/abs/2106.09004 · HTML: https://mlsanity.com/paper/2106.09004/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 79 references that are in the map.
- First posted on arXiv 10 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [Neural SDEs as Infinite-Dimensional GANs](https://mlsanity.com/paper/2102.03657/) (2021)
- [Neural Rough Differential Equations for Long Time Series](https://mlsanity.com/paper/2009.08295/) (2020)
- [Transformers for Modeling Physical Systems](https://mlsanity.com/paper/2010.03957/) (2020)
- [Data-driven approximation of the Koopman generator: Model reduction, system identification, and control](https://mlsanity.com/paper/1909.10638/) (2019)
- [Learning Poisson systems and trajectories of autonomous systems via Poisson neural networks](https://mlsanity.com/paper/2012.03133/) (2020)
- [The Robust Manifold Defense: Adversarial Training using Generative Models](https://mlsanity.com/paper/1712.09196/) (2017)
- [Learning Stochastic Differential Equations With Gaussian Processes Without Gradient Matching](https://mlsanity.com/paper/1807.05748/) (2018)
- [Identifying Latent Stochastic Differential Equations](https://mlsanity.com/paper/2007.06075/) (2020)
- [RODE-Net: Learning Ordinary Differential Equations with Randomness from Data](https://mlsanity.com/paper/2006.02377/) (2020)
- [Generative Ensemble Regression: Learning Particle Dynamics from Observations of Ensembles with Physics-Informed Deep Generative Models](https://mlsanity.com/paper/2008.01915/) (2020)

## Built on by (most cited first, 30 of 39)

- [Learning Stochastic Dynamical System via Flow Map Operator](https://mlsanity.com/paper/2305.03874/) (2023)
- [Constructing Custom Thermodynamics Using Deep Learning](https://mlsanity.com/paper/2308.04119/) (2023)
- [RandONet: Shallow-Networks with Random Projections for learning linear and nonlinear operators](https://mlsanity.com/paper/2406.05470/) (2024)
- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)
- [Nonlinear Discrete-Time Observers with Physics-Informed Neural Networks](https://mlsanity.com/paper/2402.12360/) (2024)
- [DynGMA: a robust approach for learning stochastic differential equations from data](https://mlsanity.com/paper/2402.14475/) (2024)
- [Modeling Unknown Stochastic Dynamical System Subject to External Excitation](https://mlsanity.com/paper/2406.15747/) (2024)
- [Discovering Symbolic Differential Equations with Symmetry Invariants](https://mlsanity.com/paper/2505.12083/) (2025)
- [Data-driven Effective Modeling of Multiscale Stochastic Dynamical Systems](https://mlsanity.com/paper/2408.14821/) (2024)
- [Some of the variables, some of the parameters, some of the times, with some physics known: Identification with partial information](https://mlsanity.com/paper/2304.14214/) (2023)
- [Early warning indicators via latent stochastic dynamical systems](https://mlsanity.com/paper/2309.03842/) (2023)
- [Tasks Makyth Models: Machine Learning Assisted Surrogates for Tipping Points](https://mlsanity.com/paper/2309.14334/) (2023)
- [Identifiable learning of dissipative dynamics](https://mlsanity.com/paper/2510.24160/) (2025)
- [Reservoir Computing with Error Correction: Long-term Behaviors of Stochastic Dynamical Systems](https://mlsanity.com/paper/2305.00669/) (2023)
- [MD-NOMAD: Mixture density nonlinear manifold decoder for emulating stochastic differential equations and uncertainty propagation](https://mlsanity.com/paper/2404.15731/) (2024)
- [SPIEDiff: robust learning of long-time macroscopic dynamics from short-time particle simulations with quantified epistemic uncertainty](https://mlsanity.com/paper/2505.13501/) (2025)
- [Next Generation Equation-Free Multiscale Modelling of Crowd Dynamics via Machine Learning](https://mlsanity.com/paper/2508.03926/) (2025)
- [Geometric regularization of autoencoders via observed stochastic dynamics](https://mlsanity.com/paper/2604.16282/) (2026)
- [Tipping Points of Evolving Epidemiological Networks: Machine Learning-Assisted, Data-Driven Effective Modeling](https://mlsanity.com/paper/2311.00797/) (2023)
- [Learning Stochastic Hamiltonian Systems via Stochastic Generating Function Neural Network](https://mlsanity.com/paper/2507.14467/) (2025)
- [Moment Estimate and Variational Approach for Learning Generalized Diffusion with Non-gradient Structures](https://mlsanity.com/paper/2508.01854/) (2025)
- [Structure-Aware Variational Learning of a Class of Generalized Diffusions](https://mlsanity.com/paper/2604.20188/) (2026)
- [Learning dynamical systems with biochemically informed neural ordinary differential equations](https://mlsanity.com/paper/2605.24170/) (2026)
- [Neural Langevin Dynamics: towards interpretable Neural Stochastic Differential Equations](https://mlsanity.com/paper/2211.09537/) (2022)
- [From Noise to Signal: Unveiling Treatment Effects from Digital Health Data through Pharmacology-Informed Neural-SDE](https://mlsanity.com/paper/2403.03274/) (2024)
- [Stochastic parameter reduced-order model based on hybrid machine learning approaches](https://mlsanity.com/paper/2403.17032/) (2024)
- [Feynman-Kac Operator Expectation Estimator](https://mlsanity.com/paper/2407.02010/) (2024)
- [A Robust Model-Based Approach for Continuous-Time Policy Evaluation with Unknown L'evy Process Dynamics](https://mlsanity.com/paper/2504.01482/) (2025)
- [Stochastic generative methods for stable and accurate closure modeling of chaotic dynamical systems](https://mlsanity.com/paper/2504.09750/) (2025)
- [An Adaptive Random Fourier Features approach Applied to Learning Stochastic Differential Equations](https://mlsanity.com/paper/2507.15442/) (2025)

## Abstract (arXiv)

We identify effective stochastic differential equations (SDE) for coarse observables of fine-grained particle- or agent-based simulations; these SDE then provide useful coarse surrogate models of the fine scale dynamics. We approximate the drift and diffusivity functions in these effective SDE through neural networks, which can be thought of as effective stochastic ResNets. The loss function is inspired by, and embodies, the structure of established stochastic numerical integrators (here, Euler-Maruyama and Milstein); our approximations can thus benefit from backward error analysis of these underlying numerical schemes. They also lend themselves naturally to "physics-informed" gray-box identification when approximate coarse models, such as mean field equations, are available. Existing numerical integration schemes for Langevin-type equations and for stochastic partial differential equations (SPDE) can also be used for training; we demonstrate this on a stochastically forced oscillator and the stochastic wave equation. Our approach does not require long trajectories, works on scattered snapshot data, and is designed to naturally handle different time steps per snapshot. We consider both the case where the coarse collective observables are known in advance, as well as the case where they must be found in a data-driven manner.