# Approximate Inference via Clustering

arXiv: https://arxiv.org/abs/2111.14219 · HTML: https://mlsanity.com/paper/2111.14219/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 28 Nov 2021.

## Builds on (most cited first, 11 of 11)

- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [COPOD: Copula-Based Outlier Detection](https://mlsanity.com/paper/2009.09463/) (2020)
- [Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks](https://mlsanity.com/paper/1512.07666/) (2015)
- [Practical and Optimal LSH for Angular Distance](https://mlsanity.com/paper/1509.02897/) (2015)
- [MIMOSA: Multi-constraint Molecule Sampling for Molecule Optimization](https://mlsanity.com/paper/2010.02318/) (2020)
- [Stochastic Expectation Propagation](https://mlsanity.com/paper/1506.04132/) (2015)
- [Accelerating Minibatch Stochastic Gradient Descent using Stratified Sampling](https://mlsanity.com/paper/1405.3080/) (2014)
- [Network Representation Learning: From Preprocessing, Feature Extraction to Node Embedding](https://mlsanity.com/paper/2110.07582/) (2021)
- [Bridging the Gap between Stochastic Gradient MCMC and Stochastic Optimization](https://mlsanity.com/paper/1512.07962/) (2015)
- [Exploiting the Structure: Stochastic Gradient Methods Using Raw Clusters](https://mlsanity.com/paper/1602.02151/) (2016)
- [Bitcoin Transaction Forecasting with Deep Network Representation Learning](https://mlsanity.com/paper/2007.07993/) (2020)

## Abstract (arXiv)

In recent years, large-scale Bayesian learning draws a great deal of attention. However, in big-data era, the amount of data we face is growing much faster than our ability to deal with it. Fortunately, it is observed that large-scale datasets usually own rich internal structure and is somewhat redundant. In this paper, we attempt to simplify the Bayesian posterior via exploiting this structure. Specifically, we restrict our interest to the so-called well-clustered datasets and construct an approximate posterior according to the clustering information. Fortunately, the clustering structure can be efficiently obtained via a particular clustering algorithm. When constructing the approximate posterior, the data points in the same cluster are all replaced by the centroid of the cluster. As a result, the posterior can be significantly simplified. Theoretically, we show that under certain conditions the approximate posterior we construct is close (measured by KL divergence) to the exact posterior. Furthermore, thorough experiments are conducted to validate the fact that the constructed posterior is a good approximation to the true posterior and much easier to sample from.