# Importance Weighted Hierarchical Variational Inference

arXiv: https://arxiv.org/abs/1905.03290 · HTML: https://mlsanity.com/paper/1905.03290/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2019.

## Built on by (most cited first, 20 of 20)

- [Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation](https://mlsanity.com/paper/2404.04057/) (2024)
- [GFlowNets and variational inference](https://mlsanity.com/paper/2210.00580/) (2022)
- [Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo](https://mlsanity.com/paper/2404.17546/) (2024)
- [Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Acceleration](https://mlsanity.com/paper/2310.17153/) (2023)
- [Latent State Marginalization as a Low-cost Approach for Improving Exploration](https://mlsanity.com/paper/2210.00999/) (2022)
- [Continual Variational Autoencoder Learning via Online Cooperative Memorization](https://mlsanity.com/paper/2207.10131/) (2022)
- [Recursive Monte Carlo and Variational Inference with Auxiliary Variables](https://mlsanity.com/paper/2203.02836/) (2022)
- [Kernel Semi-Implicit Variational Inference](https://mlsanity.com/paper/2405.18997/) (2024)
- [Reinforced sequential Monte Carlo for amortised sampling](https://mlsanity.com/paper/2510.11711/) (2025)
- [Lifelong Generative Modelling Using Dynamic Expansion Graph Model](https://mlsanity.com/paper/2112.08370/) (2021)
- [Probabilistic Programming with Programmable Variational Inference](https://mlsanity.com/paper/2406.15742/) (2024)
- [Bayesian Experimental Design for Model Discrepancy Calibration: An Auto-Differentiable Ensemble Kalman Inversion Approach](https://mlsanity.com/paper/2504.20319/) (2025)
- [Higher-Order Generalization Bounds: Learning Deep Probabilistic Programs via PAC-Bayes Objectives](https://mlsanity.com/paper/2203.15972/) (2022)
- [Variational Bayesian Phylogenetic Inference with Semi-implicit Branch Length Distributions](https://mlsanity.com/paper/2408.05058/) (2024)
- [A Kernel Approach for Semi-implicit Variational Inference](https://mlsanity.com/paper/2601.12023/) (2026)
- [Towards Model-Agnostic Posterior Approximation for Fast and Accurate Variational Autoencoders](https://mlsanity.com/paper/2403.08941/) (2024)
- [Revised Regularization for Efficient Continual Learning through Correlation-Based Parameter Update in Bayesian Neural Networks](https://mlsanity.com/paper/2411.14202/) (2024)
- [Revisiting Unbiased Implicit Variational Inference](https://mlsanity.com/paper/2506.03839/) (2025)
- [Semi-Implicit Variational Inference via Kernelized Path Gradient Descent](https://mlsanity.com/paper/2506.05088/) (2025)
- [Bayesian Experimental Design for Model Discrepancy Calibration: A Rivalry between Kullback--Leibler Divergence and Wasserstein Distance](https://mlsanity.com/paper/2601.16425/) (2026)

## Abstract (arXiv)

Variational Inference is a powerful tool in the Bayesian modeling toolkit, however, its effectiveness is determined by the expressivity of the utilized variational distributions in terms of their ability to match the true posterior distribution. In turn, the expressivity of the variational family is largely limited by the requirement of having a tractable density function. To overcome this roadblock, we introduce a new family of variational upper bounds on a marginal log density in the case of hierarchical models (also known as latent variable models). We then give an upper bound on the Kullback-Leibler divergence and derive a family of increasingly tighter variational lower bounds on the otherwise intractable standard evidence lower bound for hierarchical variational distributions, enabling the use of more expressive approximate posteriors. We show that previously known methods, such as Hierarchical Variational Models, Semi-Implicit Variational Inference and Doubly Semi-Implicit Variational Inference can be seen as special cases of the proposed approach, and empirically demonstrate superior performance of the proposed method in a set of experiments.