# A Group-Theoretic Framework for Data Augmentation

arXiv: https://arxiv.org/abs/1907.10905 · HTML: https://mlsanity.com/paper/1907.10905/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2019.

## Built on by (most cited first, 14 of 14)

- [Model-Based Domain Generalization](https://mlsanity.com/paper/2102.11436/) (2021)
- [Reconstruction for Powerful Graph Representations](https://mlsanity.com/paper/2110.00577/) (2021)
- [The Benefits of Mixup for Feature Learning](https://mlsanity.com/paper/2303.08433/) (2023)
- [Learning Instance-Specific Augmentations by Capturing Local Invariances](https://mlsanity.com/paper/2206.00051/) (2022)
- [Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environment](https://mlsanity.com/paper/2406.13371/) (2024)
- [In What Ways Are Deep Neural Networks Invariant and How Should We Measure This?](https://mlsanity.com/paper/2210.03773/) (2022)
- [Quantifying the Variability Collapse of Neural Networks](https://mlsanity.com/paper/2306.03440/) (2023)
- [Tilt your Head: Activating the Hidden Spatial-Invariance of Classifiers](https://mlsanity.com/paper/2405.03730/) (2024)
- [Learning Continuous Rotation Canonicalization with Radial Beam Sampling](https://mlsanity.com/paper/2206.10690/) (2022)
- [Data-Efficient Augmentation for Training Neural Networks](https://mlsanity.com/paper/2210.08363/) (2022)
- [Provable Benefit of Cutout and CutMix for Feature Learning](https://mlsanity.com/paper/2410.23672/) (2024)
- [Generalization Through the Lens of Learning Dynamics](https://mlsanity.com/paper/2212.05377/) (2022)
- [Unproportional mosaicing](https://mlsanity.com/paper/2303.02081/) (2023)
- [Randomized Dimension Reduction with Statistical Guarantees](https://mlsanity.com/paper/2310.01739/) (2023)

## Abstract (arXiv)

Data augmentation is a widely used trick when training deep neural networks: in addition to the original data, properly transformed data are also added to the training set. However, to the best of our knowledge, a clear mathematical framework to explain the performance benefits of data augmentation is not available. In this paper, we develop such a theoretical framework. We show data augmentation is equivalent to an averaging operation over the orbits of a certain group that keeps the data distribution approximately invariant. We prove that it leads to variance reduction. We study empirical risk minimization, and the examples of exponential families, linear regression, and certain two-layer neural networks. We also discuss how data augmentation could be used in problems with symmetry where other approaches are prevalent, such as in cryo-electron microscopy (cryo-EM).