mlsanityData to 7 Oct 2026

Firefly Monte Carlo: Exact MCMC with Subsets of Data

arXiv 1403.5693 ↗First posted 22 Mar 2014stat.ML cs.LG stat.COcited by 23

Not on the map. Why?

Abstract (arXiv)

Markov chain Monte Carlo (MCMC) is a popular and successful general-purpose tool for Bayesian inference. However, MCMC cannot be practically applied to large data sets because of the prohibitive cost of evaluating every likelihood term at every iteration. Here we present Firefly Monte Carlo (FlyMC) an auxiliary variable MCMC algorithm that only queries the likelihoods of a potentially small subset of the data at each iteration yet simulates from the exact posterior distribution, in contrast to recent proposals that are approximate even in the asymptotic limit. FlyMC is compatible with a wide variety of modern MCMC algorithms, and only requires a lower bound on the per-datum likelihood factors. In experiments, we find that FlyMC generates samples from the posterior more than an order of magnitude faster than regular MCMC, opening up MCMC methods to larger datasets than were previously considered feasible.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Mar 2014.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 5; 2023: 7; 2024: 3; 2025: 3.

23 papers.

Sort:

  1. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  2. 2021A Survey of Monte Carlo Methods for Parameter Estimationcited by 23
  3. 2021Surrogate Likelihoods for Variational Annealed Importance Samplingcited by 14
  4. 2022Bayesian inference via sparse Hamiltonian flowscited by 12
  5. 2021Mitigating belief projection in explainable artificial intelligence via Bayesian Teachingcited by 10
  6. 2022Fast Bayesian Coresets via Subsampling and Quasi-Newton Refinementcited by 9
  7. 2024Energy based diffusion generator for efficient sampling of Boltzmann distributionscited by 7
  8. 2023Entropy-MCMC: Sampling from Flat Basins with Easecited by 6
  9. 2023Variational Inference for Bayesian Neural Networks under Model and Parameter Uncertaintycited by 5
  10. 2023Coreset Markov Chain Monte Carlocited by 5
  11. 2023Bayesian Pseudo-Coresets via Contrastive Divergencecited by 4
  12. 2023Machine Learning and the Future of Bayesian Computationcited by 4
11 more
  1. 2023Structure Learning with Adaptive Random Neighborhood Informed MCMCcited by 4
  2. 2024General bounds on the quality of Bayesian coresetscited by 2
  3. 2021On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approachcited by 1
  4. 2022CEDAR: Communication Efficient Distributed Analysis for Regressionscited by 1
  5. 2025Bayesian Computation in Deep Learningcited by 1
  6. 2021How To Train Your Program: a Probabilistic Programming Pattern for Bayesian Learning From Datacited by 0
  7. 2022Cyclical Kernel Adaptive Metropoliscited by 0
  8. 2023The surrogate Gibbs-posterior of a corrected stochastic MALA: Towards uncertainty quantification for neural networkscited by 0
  9. 2024Markov chain Monte Carlo without evaluating the target: an auxiliary variable approachcited by 0
  10. 2025Efficient MCMC Sampling with Expensive-to-Compute and Irregular Likelihoodscited by 0
  11. 2025Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensemblescited by 0