# An information theoretic limit to data amplification

arXiv: https://arxiv.org/abs/2412.18041 · HTML: https://mlsanity.com/paper/2412.18041/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 32 references that are in the map.
- First posted on arXiv 23 Dec 2024.

## Builds on (most cited first, 7 of 7)

- [A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'](https://mlsanity.com/paper/2410.12954/) (2024)
- [CaloGAN: Simulating 3D High Energy Particle Showers in Multi-Layer Electromagnetic Calorimeters with Generative Adversarial Networks](https://mlsanity.com/paper/1712.10321/) (2017)
- [Learning Particle Physics by Example: Location-Aware Generative Adversarial Networks for Physics Synthesis](https://mlsanity.com/paper/1701.05927/) (2017)
- [A review of Generative Adversarial Networks (GANs) and its applications in a wide variety of disciplines -- From Medical to Remote Sensing](https://mlsanity.com/paper/2110.01442/) (2021)
- [The Shannon Entropy of a Histogram](https://mlsanity.com/paper/2210.02848/) (2022)
- [Generative Adversarial Networks (GAN) for compact beam source modelling in Monte Carlo simulations](https://mlsanity.com/paper/1907.13324/) (2019)
- [Encoding large scale cosmological structure with Generative Adversarial Networks](https://mlsanity.com/paper/2011.05244/) (2020)

## Built on by (most cited first, 2 of 2)

- [Forecasting Generative Amplification](https://mlsanity.com/paper/2509.08048/) (2025)
- [Generative Models and Statistical Validation](https://mlsanity.com/paper/2605.30453/) (2026)

## Abstract (arXiv)

In recent years generative artificial intelligence has been used to create data to support science analysis. For example, Generative Adversarial Networks (GANs) have been trained using Monte Carlo simulated input and then used to generate data for the same problem. This has the advantage that a GAN creates data in a significantly reduced computing time. N training events for a GAN can result in GN generated events with the gain factor, G, being more than one. This appears to violate the principle that one cannot get information for free. This is not the only way to amplify data so this process will be referred to as data amplification which is studied using information theoretic concepts. It is shown that a gain of greater than one is possible whilst keeping the information content of the data unchanged. This leads to a mathematical bound which only depends on the number of generated and training events. This study determines conditions on both the underlying and reconstructed probability distributions to ensure this bound. In particular, the resolution of variables in amplified data is not improved by the process but the increase in sample size can still improve statistical significance. The bound is confirmed using computer simulation and analysis of GAN generated data from the literature.