# Learning Partially Known Stochastic Dynamics with Empirical PAC Bayes

arXiv: https://arxiv.org/abs/2006.09914 · HTML: https://mlsanity.com/paper/2006.09914/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2020.

## Built on by (most cited first, 16 of 16)

- [User-friendly introduction to PAC-Bayes bounds](https://mlsanity.com/paper/2110.11216/) (2021)
- [Predicting the impact of treatments over time with uncertainty aware neural differential equations](https://mlsanity.com/paper/2202.11987/) (2022)
- [Learning Interacting Dynamical Systems with Latent Gaussian Process ODEs](https://mlsanity.com/paper/2205.11894/) (2022)
- [PAC-Bayesian bounds for learning LTI-ss systems with input from empirical loss](https://mlsanity.com/paper/2303.16816/) (2023)
- [Motion Forecasting via Model-Based Risk Minimization](https://mlsanity.com/paper/2409.10585/) (2024)
- [Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modeling](https://mlsanity.com/paper/2602.16864/) (2026)
- [NeuralFMU: Towards Structural Integration of FMUs into Neural Networks](https://mlsanity.com/paper/2109.04351/) (2021)
- [NeuralFMU: Presenting a workflow for integrating hybrid NeuralODEs into real world applications](https://mlsanity.com/paper/2209.03933/) (2022)
- [PAC-Bayesian Soft Actor-Critic Learning](https://mlsanity.com/paper/2301.12776/) (2023)
- [Empirical PAC-Bayes Bounds for Markov Chains](https://mlsanity.com/paper/2509.20985/) (2025)
- [Sampling-Free Probabilistic Deep State-Space Models](https://mlsanity.com/paper/2309.08256/) (2023)
- [Hybrid Modeling Design Patterns](https://mlsanity.com/paper/2401.00033/) (2023)
- [Length independent generalization bounds for deep SSM architectures via Rademacher contraction and stability constraints](https://mlsanity.com/paper/2405.20278/) (2024)
- [Continual Learning of Multi-modal Dynamics with External Memory](https://mlsanity.com/paper/2203.00936/) (2022)
- [Sharpness-Aware Hybrid Model Learning for Architecture-Agnostic Parameter Estimation](https://mlsanity.com/paper/2602.06837/) (2026)
- [Minimax estimation of the pseudo-spectral gap of continuous Markov chains, with applications to empirical concentration bounds](https://mlsanity.com/paper/2610.20007/) (2026)

## Abstract (arXiv)

Neural Stochastic Differential Equations model a dynamical environment with neural nets assigned to their drift and diffusion terms. The high expressive power of their nonlinearity comes at the expense of instability in the identification of the large set of free parameters. This paper presents a recipe to improve the prediction accuracy of such models in three steps: i) accounting for epistemic uncertainty by assuming probabilistic weights, ii) incorporation of partial knowledge on the state dynamics, and iii) training the resultant hybrid model by an objective derived from a PAC-Bayesian generalization bound. We observe in our experiments that this recipe effectively translates partial and noisy prior knowledge into an improved model fit.