# Debiased Distribution Compression

arXiv: https://arxiv.org/abs/2404.12290 · HTML: https://mlsanity.com/paper/2404.12290/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2024.

## Built on by (most cited first, 8 of 8)

- [KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learning](https://mlsanity.com/paper/2512.19605/) (2025)
- [Low-Rank Thinning](https://mlsanity.com/paper/2502.12063/) (2025)
- [WildCat: Near-Linear Attention in Theory and Practice](https://mlsanity.com/paper/2602.10056/) (2026)
- [Supervised Kernel Thinning](https://mlsanity.com/paper/2410.13749/) (2024)
- [The Minimax Lower Bound of Kernel Stein Discrepancy Estimation](https://mlsanity.com/paper/2510.15058/) (2025)
- [Stationary MMD Points](https://mlsanity.com/paper/2505.20754/) (2025)
- [Minimax Lower Bounds of Kernel Discrepancy Estimation: MMD, HSIC, KSD](https://mlsanity.com/paper/2607.24235/) (2026)
- [Thinned Mean Field Langevin Dynamics](https://mlsanity.com/paper/2605.28589/) (2026)

## Abstract (arXiv)

Modern compression methods can summarize a target distribution mathbbP more succinctly than i.i.d. sampling but require access to a low-bias input sequence like a Markov chain converging quickly to mathbbP. We introduce a new suite of compression methods suitable for compression with biased input sequences. Given n points targeting the wrong distribution and quadratic time, Stein kernel thinning (SKT) returns √n equal-weighted points with widetildeO(n-1/2) maximum mean discrepancy (MMD) to mathbbP. For larger-scale compression tasks, low-rank SKT achieves the same feat in sub-quadratic time using an adaptive low-rank debiasing procedure that may be of independent interest. For downstream tasks that support simplex or constant-preserving weights, Stein recombination and Stein Cholesky achieve even greater parsimony, matching the guarantees of SKT with as few as poly-log(n) weighted points. Underlying these advances are new guarantees for the quality of simplex-weighted coresets, the spectral decay of kernel matrices, and the covering numbers of Stein kernel Hilbert spaces. In our experiments, our techniques provide succinct and accurate posterior summaries while overcoming biases due to burn-in, approximate Markov chain Monte Carlo, and tempering.