# Variational Tempering

arXiv: https://arxiv.org/abs/1411.1810 · HTML: https://mlsanity.com/paper/1411.1810/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2014.

## Built on by (most cited first, 20 of 20)

- [Annealed Stein Variational Gradient Descent](https://mlsanity.com/paper/2101.09815/) (2021)
- [A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning](https://mlsanity.com/paper/2102.13176/) (2021)
- [On Stein Variational Neural Network Ensembles](https://mlsanity.com/paper/2106.10760/) (2021)
- [Tackling covariate shift with node-based Bayesian neural networks](https://mlsanity.com/paper/2206.02435/) (2022)
- [Dynamic Slate Recommendation with Gated Recurrent Units and Thompson Sampling](https://mlsanity.com/paper/2104.15046/) (2021)
- [Calibrated Test-Time Guidance for Bayesian Inference](https://mlsanity.com/paper/2602.22428/) (2026)
- [A Joint Learning Approach for Semi-supervised Neural Topic Modeling](https://mlsanity.com/paper/2204.03208/) (2022)
- [Variational Routing: A Scalable Bayesian Framework for Calibrated Mixture-of-Experts Transformers](https://mlsanity.com/paper/2603.09453/) (2026)
- [BAM: Bayes with Adaptive Memory](https://mlsanity.com/paper/2202.02405/) (2022)
- [Learning Hyperparameters via a Data-Emphasized Variational Objective](https://mlsanity.com/paper/2502.01861/) (2025)
- [FlowVAT: Normalizing Flow Variational Inference with Affine-Invariant Tempering](https://mlsanity.com/paper/2505.10466/) (2025)
- [Cyclical Variational Bayes Monte Carlo for Efficient Multi-Modal Posterior Distributions Evaluation](https://mlsanity.com/paper/2202.11645/) (2022)
- [Projective Integral Updates for High-Dimensional Variational Inference](https://mlsanity.com/paper/2301.08374/) (2023)
- [Federated Variational Inference for Bayesian Mixture Models](https://mlsanity.com/paper/2502.12684/) (2025)
- [Tempering the Bayes Filter towards Improved Model-Based Estimation](https://mlsanity.com/paper/2512.02823/) (2025)
- [Bayesian non-parametric non-negative matrix factorization for pattern identification in environmental mixtures](https://mlsanity.com/paper/2109.12164/) (2021)
- [Constrained Reweighting of Distributions: an Optimal Transport Approach](https://mlsanity.com/paper/2310.12447/) (2023)
- [Gaussian Process Tilted Nonparametric Density Estimation using Fisher Divergence Score Matching](https://mlsanity.com/paper/2504.03485/) (2025)
- [Stochastic Variational Inference with Tuneable Stochastic Annealing](https://mlsanity.com/paper/2504.03902/) (2025)
- [On the Effect of Regularization on Nonparametric Mean-Variance Regression](https://mlsanity.com/paper/2511.22004/) (2025)

## Abstract (arXiv)

Variational inference (VI) combined with data subsampling enables approximate posterior inference over large data sets, but suffers from poor local optima. We first formulate a deterministic annealing approach for the generic class of conditionally conjugate exponential family models. This approach uses a decreasing temperature parameter which deterministically deforms the objective during the course of the optimization. A well-known drawback to this annealing approach is the choice of the cooling schedule. We therefore introduce variational tempering, a variational algorithm that introduces a temperature latent variable to the model. In contrast to related work in the Markov chain Monte Carlo literature, this algorithm results in adaptive annealing schedules. Lastly, we develop local variational tempering, which assigns a latent temperature to each data point; this allows for dynamic annealing that varies across data. Compared to the traditional VI, all proposed approaches find improved predictive likelihoods on held-out data.