# Sample Amplification: Increasing Dataset Size even when Learning is Impossible

arXiv: https://arxiv.org/abs/1904.12053 · HTML: https://mlsanity.com/paper/1904.12053/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2019.

## Built on by (most cited first, 6 of 6)

- [Deep Generative Models for Detector Signature Simulation: A Taxonomic Review](https://mlsanity.com/paper/2312.09597/) (2023)
- [The Chamber Ensemble Generator: Limitless High-Quality MIR Data via Generative Modeling](https://mlsanity.com/paper/2209.14458/) (2022)
- [Differentially Private Sampling from Distributions](https://mlsanity.com/paper/2211.08193/) (2022)
- [On the Statistical Complexity of Sample Amplification](https://mlsanity.com/paper/2201.04315/) (2022)
- [Curricular and Cyclical Loss for Time Series Learning Strategy](https://mlsanity.com/paper/2312.15853/) (2023)
- [Reveal-or-Obscure: A Differentially Private Sampling Algorithm for Discrete Distributions](https://mlsanity.com/paper/2504.14696/) (2025)

## Abstract (arXiv)

Given data drawn from an unknown distribution, D, to what extent is it possible to ``amplify'' this dataset and output an even larger set of samples that appear to have been drawn from D? We formalize this question as follows: an (n,m) amplification procedure takes as input n independent draws from an unknown distribution D, and outputs a set of m > n ``samples''. An amplification procedure is valid if no algorithm can distinguish the set of m samples produced by the amplifier from a set of m independent draws from D, with probability greater than 2/3. Perhaps surprisingly, in many settings, a valid amplification procedure exists, even when the size of the input dataset, n, is significantly less than what would be necessary to learn D to non-trivial accuracy. Specifically we consider two fundamental settings: the case where D is an arbitrary discrete distribution supported on ≤ k elements, and the case where D is a d-dimensional Gaussian with unknown mean, and fixed covariance. In the first case, we show that an left(n, n + Theta(fracn√k)right) amplifier exists. In particular, given n=O(√k) samples from D, one can output a set of m=n+1 datapoints, whose total variation distance from the distribution of m i.i.d. draws from D is a small constant, despite the fact that one would need quadratically more data, n=Theta(k), to learn D up to small constant total variation distance. In the Gaussian case, we show that an left(n,n+Theta(fracn√d )right) amplifier exists, even though learning the distribution to small constant total variation distance requires Theta(d) samples. In both the discrete and Gaussian settings, we show that these results are tight, to constant factors. Beyond these results, we formalize a number of curious directions for future research along this vein.