# Fast and Accurate Estimation of Non-Nested Binomial Hierarchical Models Using Variational Inference

arXiv: https://arxiv.org/abs/2007.12300 · HTML: https://mlsanity.com/paper/2007.12300/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2020.

## Built on by (most cited first, 3 of 3)

- [Partially factorized variational inference for high-dimensional mixed models](https://mlsanity.com/paper/2312.13148/) (2023)
- [Linear-cost unbiased posterior estimates for crossed effects and matrix factorization models via couplings](https://mlsanity.com/paper/2410.08939/) (2024)
- [Vector Copula Variational Inference and Dependent Block Posterior Approximations](https://mlsanity.com/paper/2503.01072/) (2025)

## Abstract (arXiv)

Non-linear hierarchical models are commonly used in many disciplines. However, inference in the presence of non-nested effects and on large datasets is challenging and computationally burdensome. This paper provides two contributions to scalable and accurate inference. First, I derive a new mean-field variational algorithm for estimating binomial logistic hierarchical models with an arbitrary number of non-nested random effects. Second, I propose "marginally augmented variational Bayes" (MAVB) that further improves the initial approximation through a step of Bayesian post-processing. I prove that MAVB provides a guaranteed improvement in the approximation quality at low computational cost and induces dependencies that were assumed away by the initial factorization assumptions. I apply these techniques to a study of voter behavior using a high-dimensional application of the popular approach of multilevel regression and post-stratification (MRP). Existing estimation took hours whereas the algorithms proposed run in minutes. The posterior means are well-recovered even under strong factorization assumptions. Applying MAVB further improves the approximation by partially correcting the under-estimated variance. The proposed methodology is implemented in an open source software package.