# SING: SDE Inference via Natural Gradients

arXiv: https://arxiv.org/abs/2506.17796 · HTML: https://mlsanity.com/paper/2506.17796/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 59 references that are in the map.
- First posted on arXiv 21 Jun 2025.

## Builds on (most cited first, 29 of 29)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [The Bayesian Learning Rule](https://mlsanity.com/paper/2107.04562/) (2021)
- [Conjugate-Computation Variational Inference : Converting Variational Inference in Non-Conjugate Models to Inferences in Conjugate Models](https://mlsanity.com/paper/1703.04265/) (2017)
- [Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations](https://mlsanity.com/paper/1906.02914/) (2019)
- [Natural Gradients in Practice: Non-Conjugate Variational Inference in Gaussian Process Models](https://mlsanity.com/paper/1803.09151/) (2018)
- [Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models](https://mlsanity.com/paper/1807.04489/) (2018)
- [Towards Scalable and Stable Parallelization of Nonlinear RNNs](https://mlsanity.com/paper/2407.19115/) (2024)
- [Temporal Parallelization of Bayesian Smoothers](https://mlsanity.com/paper/1905.13002/) (2019)
- [Spatio-Temporal Variational Gaussian Processes](https://mlsanity.com/paper/2111.01732/) (2021)
- [Learning interpretable continuous-time models of latent stochastic dynamical systems](https://mlsanity.com/paper/1902.04420/) (2019)
- [Black-box Variational Inference for Stochastic Differential Equations](https://mlsanity.com/paper/1802.03335/) (2018)
- [SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equations](https://mlsanity.com/paper/2502.02472/) (2025)
- [Identifying Latent Stochastic Differential Equations](https://mlsanity.com/paper/2007.06075/) (2020)
- [Fast Variational Learning in State-Space Gaussian Process Models](https://mlsanity.com/paper/2007.04731/) (2020)
- [Dual Parameterization of Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2111.03412/) (2021)
- [Amortized Reparametrization: Efficient and Scalable Variational Inference for Latent SDEs](https://mlsanity.com/paper/2312.10550/) (2023)
- [Latent SDEs on Homogeneous Spaces](https://mlsanity.com/paper/2306.16248/) (2023)
- [Neural Structure Learning with Stochastic Differential Equations](https://mlsanity.com/paper/2311.03309/) (2023)
- [Bayesian Online Natural Gradient (BONG)](https://mlsanity.com/paper/2405.19681/) (2024)
- [Variational Gaussian Process Diffusion Processes](https://mlsanity.com/paper/2306.02066/) (2023)
- [Modeling Latent Neural Dynamics with Gaussian Process Switching Linear Dynamical Systems](https://mlsanity.com/paper/2408.03330/) (2024)
- [Variational Inference for Continuous-Time Switching Dynamical Systems](https://mlsanity.com/paper/2109.14492/) (2021)
- [Moment-Based Variational Inference for Stochastic Differential Equations](https://mlsanity.com/paper/2103.00988/) (2021)
- [Temporal Parallelization of Inference in Hidden Markov Models](https://mlsanity.com/paper/2102.05743/) (2021)
- [Linear Time GPs for Inferring Latent Trajectories from Neural Spike Trains](https://mlsanity.com/paper/2306.01802/) (2023)

## Built on by (most cited first, 5 of 5)

- [A Unifying Framework for Parallelizing Sequential Models with Linear Dynamical Systems](https://mlsanity.com/paper/2509.21716/) (2025)
- [Recursive Entropic Variational Inference for Nonlinear State-Space Models](https://mlsanity.com/paper/2511.15409/) (2025)
- [STEP: Learning STructured Embeddings for Progressive Time Series](https://mlsanity.com/paper/2605.31061/) (2026)
- [Simulation-Free Learning of GP-SDEs from Irregular Observations](https://mlsanity.com/paper/2609.33112/) (2026)
- [Gaussian Flow Dynamics: Simulation-Free Neural SDE Learning Beyond One-Time Marginals](https://mlsanity.com/paper/2610.04390/) (2026)

## Abstract (arXiv)

Latent stochastic differential equation (SDE) models are important tools for the unsupervised discovery of dynamical systems from data, with applications ranging from engineering to neuroscience. In these complex domains, exact posterior inference of the latent state path is typically intractable, motivating the use of approximate methods such as variational inference (VI). However, existing VI methods for inference in latent SDEs often suffer from slow convergence and numerical instability. We propose SDE Inference via Natural Gradients (SING), a method that leverages natural gradient VI to efficiently exploit the underlying geometry of the model and variational posterior. SING enables fast and reliable inference in latent SDE models by approximating intractable integrals and parallelizing computations in time. We provide theoretical guarantees that SING approximately optimizes the intractable, continuous-time objective of interest. Moreover, we demonstrate that better state inference enables more accurate estimation of nonlinear drift functions using, for example, Gaussian process SDE models. SING outperforms prior methods in state inference and drift estimation on a variety of datasets, including a challenging application to modeling neural dynamics in freely behaving animals. Altogether, our results illustrate the potential of SING as a tool for accurate inference in complex dynamical systems, especially those characterized by limited prior knowledge and non-conjugate structure.