# Accelerated Markov Chain Monte Carlo Using Adaptive Weighting Scheme

arXiv: https://arxiv.org/abs/2408.12888 · HTML: https://mlsanity.com/paper/2408.12888/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 49 references that are in the map.
- First posted on arXiv 23 Aug 2024.

## Builds on (most cited first, 17 of 17)

- [Machine Learning for Synthetic Data Generation: A Review](https://mlsanity.com/paper/2302.04062/) (2023)
- [Stochastic Optimization with Importance Sampling](https://mlsanity.com/paper/1401.2753/) (2014)
- [An Asynchronous Parallel Stochastic Coordinate Descent Algorithm](https://mlsanity.com/paper/1311.1873/) (2013)
- [Patterns of Scalable Bayesian Inference](https://mlsanity.com/paper/1602.05221/) (2016)
- [Approximate Inference with Amortised MCMC](https://mlsanity.com/paper/1702.08343/) (2017)
- [Asynchronous Stochastic Coordinate Descent: Parallelism and Convergence Properties](https://mlsanity.com/paper/1403.3862/) (2014)
- [Deep Learning based Multi-Label Image Classification of Protest Activities](https://mlsanity.com/paper/2301.04212/) (2023)
- [SMILES-Mamba: Chemical Mamba Foundation Models for Drug ADMET Prediction](https://mlsanity.com/paper/2408.05696/) (2024)
- [Scan Order in Gibbs Sampling: Models in Which it Matters and Bounds on How Much](https://mlsanity.com/paper/1606.03432/) (2016)
- [BioMamba: Domain-Adaptive Biomedical Language Models](https://mlsanity.com/paper/2408.02600/) (2024)
- [DrugCLIP: Contrastive Drug-Disease Interaction For Drug Repurposing](https://mlsanity.com/paper/2407.02265/) (2024)
- [Herded Gibbs Sampling](https://mlsanity.com/paper/1301.4168/) (2013)
- [Ensuring Rapid Mixing and Low Bias for Asynchronous Gibbs Sampling](https://mlsanity.com/paper/1602.07415/) (2016)
- [Faster Coordinate Descent via Adaptive Importance Sampling](https://mlsanity.com/paper/1703.02518/) (2017)
- [MambaCapsule: Towards Transparent Cardiac Disease Diagnosis with Electrocardiography Using Mamba Capsule Network](https://mlsanity.com/paper/2407.20893/) (2024)
- [Sparse Partially Collapsed MCMC for Parallel Inference in Topic Models](https://mlsanity.com/paper/1506.03784/) (2015)
- [Rapidly Mixing Gibbs Sampling for a Class of Factor Graphs Using Hierarchy Width](https://mlsanity.com/paper/1510.00756/) (2015)

## Abstract (arXiv)

Gibbs sampling is one of the most commonly used Markov Chain Monte Carlo (MCMC) algorithms due to its simplicity and efficiency. It cycles through the latent variables, sampling each one from its distribution conditional on the current values of all the other variables. Conventional Gibbs sampling is based on the systematic scan (with a deterministic order of variables). In contrast, in recent years, Gibbs sampling with random scan has shown its advantage in some scenarios. However, almost all the analyses of Gibbs sampling with the random scan are based on uniform selection of variables. In this paper, we focus on a random scan Gibbs sampling method that selects each latent variable non-uniformly. Firstly, we show that this non-uniform scan Gibbs sampling leaves the target posterior distribution invariant. Then we explore how to determine the selection probability for latent variables. In particular, we construct an objective as a function of the selection probability and solve the constrained optimization problem. We further derive an analytic solution of the selection probability, which can be estimated easily. Our algorithm relies on the simple intuition that choosing the variable updates according to their marginal probabilities enhances the mixing time of the Markov chain. Finally, we validate the effectiveness of the proposed Gibbs sampler by conducting a set of experiments on real-world applications.