# Efficiently learning and sampling multimodal distributions with data-based initialization

arXiv: https://arxiv.org/abs/2411.09117 · HTML: https://mlsanity.com/paper/2411.09117/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2024.

## Built on by (most cited first, 7 of 7)

- [Estimating Ising Models in Total Variation Distance](https://mlsanity.com/paper/2511.21008/) (2025)
- [On the Robustness of Langevin Dynamics to Score Function Error](https://mlsanity.com/paper/2603.11319/) (2026)
- [Restricted Spectral Gap Decomposition for Simulated Tempering Targeting Mixture Distributions](https://mlsanity.com/paper/2505.15059/) (2025)
- [Sampling and Identity-Testing Without Approximate Tensorization of Entropy](https://mlsanity.com/paper/2506.23456/) (2025)
- [A Hybrid Framework for Healing Semigroups with Machine Learning](https://mlsanity.com/paper/2509.01763/) (2025)
- [Scientific Data Compression and Super-Resolution Sampling](https://mlsanity.com/paper/2511.13675/) (2025)
- [A computational phase transition for learning-to-sample from Ising models](https://mlsanity.com/paper/2605.24752/) (2026)

## Abstract (arXiv)

We consider the problem of sampling a multimodal distribution with a Markov chain given a small number of samples from the stationary measure. Although mixing can be arbitrarily slow, we show that if the Markov chain has a kth order spectral gap, initialization from a set of tilde O(k/varepsilon2) samples from the stationary distribution will, with high probability over the samples, efficiently generate a sample whose conditional law is varepsilon-close in TV distance to the stationary measure. In particular, this applies to mixtures of k distributions satisfying a Poincar'e inequality, with faster convergence when they satisfy a log-Sobolev inequality. Our bounds are stable to perturbations to the Markov chain, and in particular work for Langevin diffusion over mathbb Rd with score estimation error, as well as Glauber dynamics combined with approximation error from pseudolikelihood estimation. This justifies the success of data-based initialization for score matching methods despite slow mixing for the data distribution, and improves and generalizes the results of Koehler and Vuong (2023) to have linear, rather than exponential, dependence on k and apply to arbitrary semigroups. As a consequence of our results, we show for the first time that a natural class of low-complexity Ising measures can be efficiently learned from samples.