mlsanityData to 7 Oct 2026

Neural SDEs as Infinite-Dimensional GANs

arXiv 2102.03657 ↗First posted 6 Feb 2021cs.LGcited by 159

Not on the map. Why?

Cited by 23 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Stochastic differential equations (SDEs) are a staple of mathematical modelling of temporal dynamics. However, a fundamental limitation has been that such models have typically been relatively inflexible, which recent work introducing Neural SDEs has sought to solve. Here, we show that the current classical approach to fitting SDEs may be approached as a special case of (Wasserstein) GANs, and in doing so the neural and classical regimes may be brought together. The input noise is Brownian motion, the output samples are time-evolving paths produced by a numerical solver, and by parameterising a discriminator as a Neural Controlled Differential Equation (CDE), we obtain Neural SDEs as (in modern machine learning parlance) continuous-time generative time series models. Unlike previous work on this problem, this is a direct extension of the classical approach without reference to either prespecified statistics or density functions. Arbitrary drift and diffusions are admissible, so as the Wasserstein loss has a unique global minima, in the infinite data limit any SDE may be learnt. Example code has been made available as part of the texttttorchsde repository.

Key facts. Cited by 159 papers in the map (data through 7 Oct 2026). Builds on 38 of its 69 references that are in the map. First posted on arXiv 6 Feb 2021.

Builds on

38 of 69 references.

References · 38

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2018Neural Ordinary Differential Equationscited by 3,733
  7. 2017Improved Training of Wasserstein GANscited by 2,223
  8. 2015Variational Inference with Normalizing Flowscited by 2,161
  9. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  10. 2018Demystifying MMD GANscited by 1,268
  11. 2019Deep Equilibrium Modelscited by 603
  12. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
26 more
  1. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  2. 2020Lagrangian Neural Networkscited by 388
  3. 2020Universal Differential Equations for Scientific Machine Learningcited by 345
  4. 2017Real-valued (Medical) Time Series Generation with Recurrent Conditional GANscited by 329
  5. 2015Generative Moment Matching Networkscited by 252
  6. 2017MMD GAN: Towards Deeper Understanding of Moment Matching Networkcited by 251
  7. 2019Universal Approximation with Deep Narrow Networkscited by 198
  8. 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 168
  9. 2020Stochastic Normalizing Flowscited by 156
  10. 2020Dissecting Neural ODEscited by 150
  11. 2020Multiscale Deep Equilibrium Modelscited by 149
  12. 2019Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noisecited by 98
  13. 2019Theoretical guarantees for sampling and inference in generative models with latent diffusionscited by 97
  14. 2016Kernels for sequentially ordered datacited by 83
  15. 2019A Differentiable Programming System to Bridge Machine Learning and Scientific Computingcited by 81
  16. 2018The Unusual Effectiveness of Averaging in GAN Trainingcited by 73
  17. 2020On Second Order Behaviour in Augmented Neural ODEscited by 68
  18. 2017Learning to Detect Sepsis with a Multitask Gaussian Process RNN Classifiercited by 47
  19. 2020Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPUcited by 45
  20. 2019Statistical Inference for Generative Models with Maximum Mean Discrepancycited by 43
  21. 2020Modeling Continuous Stochastic Processes with Dynamic Normalizing Flowscited by 37
  22. 2020A generative adversarial network approach to calibration of local stochastic volatility modelscited by 28
  23. 2016A scalable end-to-end Gaussian process adapter for irregularly sampled time series classificationcited by 27
  24. 2020Robust pricing and hedging via neural SDEscited by 21
  25. 2020Stochasticity in Neural ODEs: An Empirical Studycited by 13
  26. 2019Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariancescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 14; 2023: 30; 2024: 37; 2025: 42; 2026: 24.

159 papers.

Sort:

By area:

  1. 2022Riemannian Diffusion Modelscited by 120
  2. 2021Modeling Irregular Time Series with Continuous Recurrent Unitscited by 101
  3. 2023Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecastingcited by 86
  4. 2021Efficient and Accurate Gradients for Neural SDEscited by 68
  5. 2022Manifold Interpolating Optimal-Transport Flows for Trajectory Inferencecited by 63
  6. 2023Functional Flow Matchingcited by 63
  7. 2024Probabilistic Forecasting with Stochastic Interpolants and F"ollmer Processescited by 61
  8. 2022Generative Adversarial Neural Operatorscited by 51
  9. 2021Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamicscited by 46
  10. 2024Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Seriescited by 42
  11. 2022Modeling Temporal Data as Continuous Functions with Stochastic Process Diffusioncited by 39
  12. 2023Stochastic Optimal Control Matchingcited by 39
38 more
  1. 2021Learning effective stochastic differential equations from microscopic simulations: linking stochastic numerics to deep learningcited by 37
  2. 2022Deep Latent State Space Models for Time-Series Generationcited by 37
  3. 2023Non-adversarial training of Neural SDEs with signature kernel scorescited by 36
  4. 2024Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Datacited by 35
  5. 2022Signature Methods in Machine Learningcited by 33
  6. 2023Physics-Informed Machine Learning for Modeling and Control of Dynamical Systemscited by 28
  7. 2023TSGBench: Time Series Generation Benchmarkcited by 24
  8. 2022Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformercited by 22
  9. 2024Trajectory Flow Matching with Applications to Clinical Time Series Modelingcited by 21
  10. 2025Comprehensive Review of Neural Differential Equations for Time Series Analysiscited by 20
  11. 2023Global universal approximation of functional input maps on weighted spacescited by 19
  12. 2024ImageFlowNet: Forecasting Multiscale Image-Level Trajectories of Disease Progression with Irregularly-Sampled Longitudinal Medical Imagescited by 19
  13. 2025From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster trainingcited by 17
  14. 2025Curly Flow Matching for Learning Non-gradient Field Dynamicscited by 17
  15. 2023Latent SDEs on Homogeneous Spacescited by 16
  16. 2023Solving Inverse Physics Problems with Score Matchingcited by 15
  17. 2023A Brief Survey on the Approximation Theory for Sequence Modellingcited by 15
  18. 2023Amortized Reparametrization: Efficient and Scalable Variational Inference for Latent SDEscited by 15
  19. 2021Scalable Inference in SDEs by Direct Matching of the Fokker-Planck-Kolmogorov Equationcited by 14
  20. 2024Fixed Point Diffusion Modelscited by 14
  21. 2023PCF-GAN: generating sequential data via the characteristic function of measures on the path spacecited by 13
  22. 2023Neural Ideal Large Eddy Simulation: Modeling Turbulence with Neural Stochastic Differential Equationscited by 13
  23. 2024Lecture notes on rough paths and applications to machine learningcited by 13
  24. 2024Geometric Trajectory Diffusion Modelscited by 13
  25. 2023New directions in the applications of rough path theorycited by 12
  26. 2023STDiff: Spatio-temporal Diffusion for Continuous Stochastic Video Predictioncited by 12
  27. 2021Physics-Augmented Learning: A New Paradigm Beyond Physics-Informed Learningcited by 11
  28. 2024A Taxonomy of Loss Functions for Stochastic Optimal Controlcited by 11
  29. 2025Multi-Marginal Stochastic Flow Matching for High-Dimensional Snapshot Data at Irregular Time Pointscited by 11
  30. 2021Quantum Quantile Mechanics: Solving Stochastic Differential Equations for Generating Time-Seriescited by 10
  31. 2022Robotic Table Wiping via Reinforcement Learning and Whole-body Trajectory Optimizationcited by 10
  32. 2023How to Learn and Generalize From Three Minutes of Data: Physics-Constrained and Uncertainty-Aware Neural Stochastic Differential Equationscited by 10
  33. 2022Fractional SDE-Net: Generation of Time Series Data with Long-term Memorycited by 9
  34. 2023Random Fourier Signature Featurescited by 9
  35. 2024Exact Gradients for Stochastic Spiking Neural Networks Driven by Rough Signalscited by 9
  36. 2024Efficient, Accurate and Stable Gradients for Neural ODEscited by 9
  37. 2023The Signature Kernelcited by 8
  38. 2024Parametric model reduction of mean-field and stochastic systems via higher-order action matchingcited by 8

50 most cited of 159 All 159 (JSON)