# Amortized Reparametrization: Efficient and Scalable Variational Inference for Latent SDEs

arXiv: https://arxiv.org/abs/2312.10550 · HTML: https://mlsanity.com/paper/2312.10550/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 16 Dec 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [Neural SDEs as Infinite-Dimensional GANs](https://mlsanity.com/paper/2102.03657/) (2021)
- [Hamiltonian Generative Networks](https://mlsanity.com/paper/1909.13789/) (2019)
- [Learning Differential Equations that are Easy to Solve](https://mlsanity.com/paper/2007.04504/) (2020)
- [Learning unknown ODE models with Gaussian processes](https://mlsanity.com/paper/1803.04303/) (2018)
- [Deep Temporal Sigmoid Belief Networks for Sequence Modeling](https://mlsanity.com/paper/1509.07087/) (2015)
- [Weak Form Generalized Hamiltonian Learning](https://mlsanity.com/paper/2104.05096/) (2021)
- [Exactly Sparse Gaussian Variational Inference with Application to Derivative-Free Batch Nonlinear State Estimation](https://mlsanity.com/paper/1911.08333/) (2019)

## Built on by (most cited first, 18 of 18)

- [SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equations](https://mlsanity.com/paper/2502.02472/) (2025)
- [Modeling Latent Neural Dynamics with Gaussian Process Switching Linear Dynamical Systems](https://mlsanity.com/paper/2408.03330/) (2024)
- [SING: SDE Inference via Natural Gradients](https://mlsanity.com/paper/2506.17796/) (2025)
- [Neural Stochastic Flows: Solver-Free Modelling and Inference for SDE Solutions](https://mlsanity.com/paper/2510.25769/) (2025)
- [Generative Modeling of Neural Dynamics via Latent Stochastic Differential Equations](https://mlsanity.com/paper/2412.12112/) (2024)
- [Hierarchical Testing with Rabbit Optimization for Industrial Cyber-Physical Systems](https://mlsanity.com/paper/2507.04100/) (2025)
- [Efficient Training of Neural SDEs Using Stochastic Optimal Control](https://mlsanity.com/paper/2505.17150/) (2025)
- [Learning Stochastic Multiscale Models](https://mlsanity.com/paper/2506.22655/) (2025)
- [Diffusion differentiable resampling](https://mlsanity.com/paper/2512.10401/) (2025)
- [HGAN-SDEs: Learning Neural Stochastic Differential Equations with Hermite-Guided Adversarial Training](https://mlsanity.com/paper/2512.20272/) (2025)
- [First-Order Trajectory Matching: Fast Ensemble Predictions of Chaotic, Turbulent, Stochastic Systems](https://mlsanity.com/paper/2606.11138/) (2026)
- [Data-driven stochastic reduced-order modeling of parametrized dynamical systems](https://mlsanity.com/paper/2601.10690/) (2026)
- [Scalable Gaussian process modeling of parametrized spatio-temporal fields](https://mlsanity.com/paper/2603.00290/) (2026)
- [Continuous-Time Trajectory Generation from Discrete Observations with Stochasticity](https://mlsanity.com/paper/2609.32026/) (2026)
- [Simulation-Free Learning of GP-SDEs from Irregular Observations](https://mlsanity.com/paper/2609.33112/) (2026)
- [Gaussian Flow Dynamics: Simulation-Free Neural SDE Learning Beyond One-Time Marginals](https://mlsanity.com/paper/2610.04390/) (2026)
- [Scan-Parallel Variational Inference for event-driven Latent Jump Models](https://mlsanity.com/paper/2610.12986/) (2026)
- [Decoupled Drift-Diffusion World Models for Consistent Future Frame Prediction](https://mlsanity.com/paper/2610.19212/) (2026)

## Abstract (arXiv)

We consider the problem of inferring latent stochastic differential equations (SDEs) with a time and memory cost that scales independently with the amount of data, the total length of the time series, and the stiffness of the approximate differential equations. This is in stark contrast to typical methods for inferring latent differential equations which, despite their constant memory cost, have a time complexity that is heavily dependent on the stiffness of the approximate differential equation. We achieve this computational advancement by removing the need to solve differential equations when approximating gradients using a novel amortization strategy coupled with a recently derived reparametrization of expectations under linear SDEs. We show that, in practice, this allows us to achieve similar performance to methods based on adjoint sensitivities with more than an order of magnitude fewer evaluations of the model in training.