# Non-adversarial training of Neural SDEs with signature kernel scores

arXiv: https://arxiv.org/abs/2305.16274 · HTML: https://mlsanity.com/paper/2305.16274/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 66 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 30 of 38)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [On Neural Differential Equations](https://mlsanity.com/paper/2202.02435/) (2022)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit](https://mlsanity.com/paper/1905.09883/) (2019)
- [Neural SDEs as Infinite-Dimensional GANs](https://mlsanity.com/paper/2102.03657/) (2021)
- [Stochastic Normalizing Flows](https://mlsanity.com/paper/2002.06707/) (2020)
- [Neural Jump Stochastic Differential Equations](https://mlsanity.com/paper/1905.10403/) (2019)
- [Neural Rough Differential Equations for Long Time Series](https://mlsanity.com/paper/2009.08295/) (2020)
- [Deep Signature Transforms](https://mlsanity.com/paper/1905.08494/) (2019)
- [Sig-Wasserstein GANs for Time Series Generation](https://mlsanity.com/paper/2111.01207/) (2021)
- [The Signature Kernel is the solution of a Goursat PDE](https://mlsanity.com/paper/2006.14794/) (2020)
- [Benchmark Dataset for Mid-Price Forecasting of Limit Order Book Data with Machine Learning Methods](https://mlsanity.com/paper/1705.03233/) (2017)
- [Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamics](https://mlsanity.com/paper/2110.10249/) (2021)
- [A Spectral Energy Distance for Parallel Speech Synthesis](https://mlsanity.com/paper/2008.01160/) (2020)
- [Get Real: Realism Metrics for Robust Limit Order Book Market Simulations](https://mlsanity.com/paper/1912.04941/) (2019)
- [Kernel Distribution Embeddings: Universal Kernels, Characteristic Kernels and Kernel Metrics on Distributions](https://mlsanity.com/paper/1604.05251/) (2016)
- [Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows](https://mlsanity.com/paper/2002.10516/) (2020)
- [Distribution Regression for Sequential Data](https://mlsanity.com/paper/2006.05805/) (2020)
- [Generating Realistic Stock Market Order Streams](https://mlsanity.com/paper/2006.04212/) (2020)
- [Towards Realistic Market Simulations: a Generative Adversarial Networks Approach](https://mlsanity.com/paper/2110.13287/) (2021)
- [A Kernel Two-Sample Test for Functional Data](https://mlsanity.com/paper/2008.11095/) (2020)
- [Higher Order Kernel Mean Embeddings to Capture Filtrations of Stochastic Processes](https://mlsanity.com/paper/2109.03582/) (2021)
- [Learning to simulate realistic limit order book markets from data as a World Agent](https://mlsanity.com/paper/2210.09897/) (2022)
- [Neural signature kernels as infinite-width-depth-limits of controlled ResNets](https://mlsanity.com/paper/2303.17671/) (2023)
- [Robust pricing and hedging via neural SDEs](https://mlsanity.com/paper/2007.04154/) (2020)

## Built on by (most cited first, 30 of 37)

- [Theoretical Foundations of Deep Selective State-Space Models](https://mlsanity.com/paper/2402.19047/) (2024)
- [Distributional Diffusion Models with Scoring Rules](https://mlsanity.com/paper/2502.02483/) (2025)
- [Signature Kernel Conditional Independence Tests in Causal Discovery for Stochastic Processes](https://mlsanity.com/paper/2402.18477/) (2024)
- [Proper scoring rules for estimation and forecast evaluation](https://mlsanity.com/paper/2504.01781/) (2025)
- [Structured Linear CDEs: Maximally Expressive and Parallel-in-Time Sequence Models](https://mlsanity.com/paper/2505.17761/) (2025)
- [Random Fourier Signature Features](https://mlsanity.com/paper/2311.12214/) (2023)
- [Exact Gradients for Stochastic Spiking Neural Networks Driven by Rough Signals](https://mlsanity.com/paper/2405.13587/) (2024)
- [Efficient, Accurate and Stable Gradients for Neural ODEs](https://mlsanity.com/paper/2410.11648/) (2024)
- [Time-Causal VAE: Robust Financial Time Series Generator](https://mlsanity.com/paper/2411.02947/) (2024)
- [Flow Straight and Fast in Hilbert Space: Functional Rectified Flow](https://mlsanity.com/paper/2509.10384/) (2025)
- [pySigLib -- Fast Signature-Based Computations on CPU and GPU](https://mlsanity.com/paper/2509.10613/) (2025)
- [Reversible Deep Equilibrium Models](https://mlsanity.com/paper/2509.12917/) (2025)
- [Universal randomised signatures for generative time series modelling](https://mlsanity.com/paper/2406.10214/) (2024)
- [Efficient Training of Neural Stochastic Differential Equations by Matching Finite Dimensional Distributions](https://mlsanity.com/paper/2410.03973/) (2024)
- [ParallelFlow: Parallelizing Linear Transformers via Flow Discretization](https://mlsanity.com/paper/2504.00492/) (2025)
- [Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modeling](https://mlsanity.com/paper/2602.16864/) (2026)
- [Signature Maximum Mean Discrepancy Two-Sample Statistical Tests](https://mlsanity.com/paper/2506.01718/) (2025)
- [Non-parametric online market regime detection and regime clustering for multidimensional and path-dependent data structures](https://mlsanity.com/paper/2306.15835/) (2023)
- [Universal approximation property of neural stochastic differential equations](https://mlsanity.com/paper/2503.16696/) (2025)
- [Deep Neural Networks Inspired by Differential Equations](https://mlsanity.com/paper/2510.09685/) (2025)
- [Signature Kernel Scoring Rule: A Spatio-Temporal Diagnostic for Probabilistic Weather Forecasting](https://mlsanity.com/paper/2510.19110/) (2025)
- [Neural stochastic Volterra equations: learning path-dependent dynamics](https://mlsanity.com/paper/2407.19557/) (2024)
- [Efficient Neural SDE Training using Wiener-Space Cubature](https://mlsanity.com/paper/2502.12395/) (2025)
- [Explicit and Effectively Symmetric Schemes for Neural SDEs on Lie Groups](https://mlsanity.com/paper/2509.20599/) (2025)
- [HGAN-SDEs: Learning Neural Stochastic Differential Equations with Hermite-Guided Adversarial Training](https://mlsanity.com/paper/2512.20272/) (2025)
- [Conservative Continuous-Time Treatment Optimization](https://mlsanity.com/paper/2603.16789/) (2026)
- [Generating Financial Time Series by Matching Random Convolutional Features](https://mlsanity.com/paper/2606.05138/) (2026)
- [Log-PDE Methods for Rough Signature Kernels](https://mlsanity.com/paper/2404.02926/) (2024)
- [Neural Stochastic Differential Equations on Compact State Spaces: Theory, Methods, and Application to Suicide Risk Modeling](https://mlsanity.com/paper/2508.17090/) (2025)
- [Novelty detection on path space](https://mlsanity.com/paper/2512.03243/) (2025)

## Abstract (arXiv)

Neural SDEs are continuous-time generative models for sequential data. State-of-the-art performance for irregular time series generation has been previously obtained by training these models adversarially as GANs. However, as typical for GAN architectures, training is notoriously unstable, often suffers from mode collapse, and requires specialised techniques such as weight clipping and gradient penalty to mitigate these issues. In this paper, we introduce a novel class of scoring rules on pathspace based on signature kernels and use them as objective for training Neural SDEs non-adversarially. By showing strict properness of such kernel scores and consistency of the corresponding estimators, we provide existence and uniqueness guarantees for the minimiser. With this formulation, evaluating the generator-discriminator pair amounts to solving a system of linear path-dependent PDEs which allows for memory-efficient adjoint-based backpropagation. Moreover, because the proposed kernel scores are well-defined for paths with values in infinite dimensional spaces of functions, our framework can be easily extended to generate spatiotemporal data. Our procedure permits conditioning on a rich variety of market conditions and significantly outperforms alternative ways of training Neural SDEs on a variety of tasks including the simulation of rough volatility models, the conditional probabilistic forecasts of real-world forex pairs where the conditioning variable is an observed past trajectory, and the mesh-free generation of limit order book dynamics.