# Filtering Variational Objectives

arXiv: https://arxiv.org/abs/1705.09279 · HTML: https://mlsanity.com/paper/1705.09279/

## Key facts

- Cited by 91 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2017.

## Built on by (most cited first, 30 of 91)

- [Score-based Data Assimilation](https://mlsanity.com/paper/2306.10574/) (2023)
- [Annealed Flow Transport Monte Carlo](https://mlsanity.com/paper/2102.07501/) (2021)
- [Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo](https://mlsanity.com/paper/2404.17546/) (2024)
- [Continual Repeated Annealed Flow Transport Monte Carlo](https://mlsanity.com/paper/2201.13117/) (2022)
- [Differentiable Particle Filtering via Entropy-Regularized Optimal Transport](https://mlsanity.com/paper/2102.07850/) (2021)
- [A Variational Perspective on Generative Flow Networks](https://mlsanity.com/paper/2210.07992/) (2022)
- [Monte Carlo Variational Auto-Encoders](https://mlsanity.com/paper/2106.15921/) (2021)
- [Differentiable Annealed Importance Sampling and the Perils of Gradient Noise](https://mlsanity.com/paper/2107.10211/) (2021)
- [Auto-differentiable Ensemble Kalman Filters](https://mlsanity.com/paper/2107.07687/) (2021)
- [An overview of differentiable particle filters for data-adaptive sequential Bayesian inference](https://mlsanity.com/paper/2302.09639/) (2023)
- [SIXO: Smoothing Inference with Twisted Objectives](https://mlsanity.com/paper/2206.05952/) (2022)
- [Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding](https://mlsanity.com/paper/2102.11086/) (2021)
- [Online Variational Filtering and Parameter Learning](https://mlsanity.com/paper/2110.13549/) (2021)
- [Differentiable Particle Filtering without Modifying the Forward Pass](https://mlsanity.com/paper/2106.10314/) (2021)
- [RNN with Particle Flow for Probabilistic Spatio-temporal Forecasting](https://mlsanity.com/paper/2106.06064/) (2021)
- [Stein Variational Gradient Descent: many-particle and long-time asymptotics](https://mlsanity.com/paper/2102.12956/) (2021)
- [Alpha-divergence Variational Inference Meets Importance Weighted Auto-Encoders: Methodology and Asymptotics](https://mlsanity.com/paper/2210.06226/) (2022)
- [Inferring stochastic low-rank recurrent neural networks from neural data](https://mlsanity.com/paper/2406.16749/) (2024)
- [Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models](https://mlsanity.com/paper/2101.07046/) (2021)
- [Nested Variational Inference](https://mlsanity.com/paper/2106.11302/) (2021)
- [Surrogate Likelihoods for Variational Annealed Importance Sampling](https://mlsanity.com/paper/2112.12194/) (2021)
- [Reduced-Order Autodifferentiable Ensemble Kalman Filters](https://mlsanity.com/paper/2301.11961/) (2023)
- [Variational Combinatorial Sequential Monte Carlo Methods for Bayesian Phylogenetic Inference](https://mlsanity.com/paper/2106.00075/) (2021)
- [On Incorporating Inductive Biases into VAEs](https://mlsanity.com/paper/2106.13746/) (2021)
- [Information is Power: Intrinsic Control via Information Capture](https://mlsanity.com/paper/2112.03899/) (2021)
- [Empirical Evaluation of Biased Methods for Alpha Divergence Minimization](https://mlsanity.com/paper/2105.06587/) (2021)
- [Normalizing Flow-based Differentiable Particle Filters](https://mlsanity.com/paper/2403.01499/) (2024)
- [On the Identifiability of Switching Dynamical Systems](https://mlsanity.com/paper/2305.15925/) (2023)
- [Adaptive Annealed Importance Sampling with Constant Rate Progress](https://mlsanity.com/paper/2306.15283/) (2023)
- [Sequential Monte Carlo for Inclusive KL Minimization in Amortized Variational Inference](https://mlsanity.com/paper/2403.10610/) (2024)

## Abstract (arXiv)

When used as a surrogate objective for maximum likelihood estimation in latent variable models, the evidence lower bound (ELBO) produces state-of-the-art results. Inspired by this, we consider the extension of the ELBO to a family of lower bounds defined by a particle filter's estimator of the marginal likelihood, the filtering variational objectives (FIVOs). FIVOs take the same arguments as the ELBO, but can exploit a model's sequential structure to form tighter bounds. We present results that relate the tightness of FIVO's bound to the variance of the particle filter's estimator by considering the generic case of bounds defined as log-transformed likelihood estimators. Experimentally, we show that training with FIVO results in substantial improvements over training the same model architecture with the ELBO on sequential data.