# Variational excess risk bound for general state space models

arXiv: https://arxiv.org/abs/2312.09607 · HTML: https://mlsanity.com/paper/2312.09607/

## Key facts

- Cited by 2 papers in the map (data through 7 Oct 2026).
- Builds on 10 of its 20 references that are in the map.
- First posted on arXiv 15 Dec 2023.

## Builds on (most cited first, 10 of 10)

- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Structured Inference Networks for Nonlinear State Space Models](https://mlsanity.com/paper/1609.09869/) (2016)
- [Validated Variational Inference via Practical Posterior Error Bounds](https://mlsanity.com/paper/1910.04102/) (2019)
- [Online Variational Filtering and Parameter Learning](https://mlsanity.com/paper/2110.13549/) (2021)
- [Variational Message Passing with Structured Inference Networks](https://mlsanity.com/paper/1803.05589/) (2018)
- [Consistency of Variational Bayes Inference for Estimation and Model Selection in Mixtures](https://mlsanity.com/paper/1805.05054/) (2018)
- [Diffusion bridges vector quantized Variational AutoEncoders](https://mlsanity.com/paper/2202.04895/) (2022)
- [Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theory](https://mlsanity.com/paper/2310.04935/) (2023)
- [A General Method for Amortizing Variational Filtering](https://mlsanity.com/paper/1811.05090/) (2018)

## Built on by (most cited first, 2 of 2)

- [Tree-based variational inference for Poisson log-normal models](https://mlsanity.com/paper/2406.17361/) (2024)
- [PAC-Bayesian Reconstruction Guarantees for Time Series Variational Autoencoders](https://mlsanity.com/paper/2609.05212/) (2026)

## Abstract (arXiv)

In this paper, we consider variational autoencoders (VAE) for general state space models. We consider a backward factorization of the variational distributions to analyze the excess risk associated with VAE. Such backward factorizations were recently proposed to perform online variational learning and to obtain upper bounds on the variational estimation error. When independent trajectories of sequences are observed and under strong mixing assumptions on the state space model and on the variational distribution, we provide an oracle inequality explicit in the number of samples and in the length of the observation sequences. We then derive consequences of this theoretical result. In particular, when the data distribution is given by a state space model, we provide an upper bound for the Kullback-Leibler divergence between the data distribution and its estimator and between the variational posterior and the estimated state space posterior distributions.Under classical assumptions, we prove that our results can be applied to Gaussian backward kernels built with dense and recurrent neural networks.