# Herded Gibbs Sampling

arXiv: https://arxiv.org/abs/1301.4168 · HTML: https://mlsanity.com/paper/1301.4168/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jan 2013.

## Built on by (most cited first, 5 of 5)

- [Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models](https://mlsanity.com/paper/2408.02085/) (2024)
- [Mixed Variational Flows for Discrete Variables](https://mlsanity.com/paper/2308.15613/) (2023)
- [Continuous Herded Gibbs Sampling](https://mlsanity.com/paper/2106.06430/) (2021)
- [Entropic Herding](https://mlsanity.com/paper/2112.11616/) (2021)
- [Accelerated Markov Chain Monte Carlo Using Adaptive Weighting Scheme](https://mlsanity.com/paper/2408.12888/) (2024)

## Abstract (arXiv)

The Gibbs sampler is one of the most popular algorithms for inference in statistical models. In this paper, we introduce a herding variant of this algorithm, called herded Gibbs, that is entirely deterministic. We prove that herded Gibbs has an O(1/T) convergence rate for models with independent variables and for fully connected probabilistic graphical models. Herded Gibbs is shown to outperform Gibbs in the tasks of image denoising with MRFs and named entity recognition with CRFs. However, the convergence for herded Gibbs for sparsely connected probabilistic graphical models is still an open problem.