# DeepBayes -- an estimator for parameter estimation in stochastic nonlinear dynamical models

arXiv: https://arxiv.org/abs/2205.02264 · HTML: https://mlsanity.com/paper/2205.02264/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 45 references that are in the map.
- First posted on arXiv 4 May 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions](https://mlsanity.com/paper/1909.00590/) (2019)
- [Visualizing and Understanding Recurrent Networks](https://mlsanity.com/paper/1506.02078/) (2015)
- [Dynamical Variational Autoencoders: A Comprehensive Review](https://mlsanity.com/paper/2008.12595/) (2020)
- [How to Construct Deep Recurrent Neural Networks](https://mlsanity.com/paper/1312.6026/) (2013)
- [On Particle Methods for Parameter Estimation in State-Space Models](https://mlsanity.com/paper/1412.8695/) (2014)
- [Deep State Space Models for Nonlinear System Identification](https://mlsanity.com/paper/2003.14162/) (2020)
- [A Gentle Tutorial of Recurrent Neural Network with Error Backpropagation](https://mlsanity.com/paper/1610.02583/) (2016)
- [Good Initializations of Variational Bayes for Deep Models](https://mlsanity.com/paper/1810.08083/) (2018)
- [A novel Deep Neural Network architecture for non-linear system identification](https://mlsanity.com/paper/2106.03078/) (2021)
- [Sequential Monte Carlo Methods for System Identification](https://mlsanity.com/paper/1503.06058/) (2015)

## Built on by (most cited first, 1 of 1)

- [Fine Tuning a Simulation-Driven Estimator](https://mlsanity.com/paper/2504.04480/) (2025)

## Abstract (arXiv)

Stochastic nonlinear dynamical systems are ubiquitous in modern, real-world applications. Yet, estimating the unknown parameters of stochastic, nonlinear dynamical models remains a challenging problem. The majority of existing methods employ maximum likelihood or Bayesian estimation. However, these methods suffer from some limitations, most notably the substantial computational time for inference coupled with limited flexibility in application. In this work, we propose DeepBayes estimators that leverage the power of deep recurrent neural networks in learning an estimator. The method consists of first training a recurrent neural network to minimize the mean-squared estimation error over a set of synthetically generated data using models drawn from the model set of interest. The a priori trained estimator can then be used directly for inference by evaluating the network with the estimation data. The deep recurrent neural network architectures can be trained offline and ensure significant time savings during inference. We experiment with two popular recurrent neural networks -- long short term memory network (LSTM) and gated recurrent unit (GRU). We demonstrate the applicability of our proposed method on different example models and perform detailed comparisons with state-of-the-art approaches. We also provide a study on a real-world nonlinear benchmark problem. The experimental evaluations show that the proposed approach is asymptotically as good as the Bayes estimator.