mlsanityData to 7 Oct 2026

Data Augmentation as Feature Manipulation

arXiv 2203.01572 ↗First posted 3 Mar 2022cs.LG stat.MLcited by 18

Not on the map. Why?

Abstract (arXiv)

Data augmentation is a cornerstone of the machine learning pipeline, yet its theoretical underpinnings remain unclear. Is it merely a way to artificially augment the data set size? Or is it about encouraging the model to satisfy certain invariance? In this work we consider another angle, and we study the effect of data augmentation on the dynamic of the learning process. We find that data augmentation can alter the relative importance of various features, effectively making certain informative but hard to learn features more likely to be captured in the learning process. Importantly, we show that this effect is more pronounced for non-linear models, such as neural networks. Our main contribution is a detailed analysis of data augmentation on the learning dynamic for a two layer convolutional neural network in the recently proposed multi-view data model by Allen-Zhu and Li [2020]. We complement this analysis with further experimental evidence that data augmentation can be viewed as feature manipulation.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 15 of its 28 references that are in the map. First posted on arXiv 3 Mar 2022.

Builds on

15 of 28 references.

References · 15

  1. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  2. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  3. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  4. 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 339
  5. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 292
  6. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 223
  7. 2018Why do deep convolutional networks generalize so poorly to small image transformations?cited by 198
  8. 2020Feature Purification: How Adversarial Training Performs Robust Deep Learningcited by 107
  9. 2018A Kernel Theory of Modern Data Augmentationcited by 106
  10. 2021Learning with invariances in random features and kernel modelscited by 89
  11. 2020A law of robustness for two-layers neural networkscited by 54
  12. 2020On the Generalization Effects of Linear Transformations in Data Augmentationcited by 47
3 more
  1. 2022Sample Efficiency of Data Augmentation Consistency Regularizationcited by 28
  2. 2019Does Data Augmentation Lead to Positive Margin?cited by 18
  3. 2020How Data Augmentation affects Optimization for Linear Regressioncited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 9; 2024: 2; 2025: 0; 2026: 1.

18 papers.

Sort:

  1. 2022Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Datacited by 46
  2. 2023Benign Overfitting for Two-layer ReLU Convolutional Neural Networkscited by 34
  3. 2023Why Does Sharpness-Aware Minimization Generalize Better Than SGD?cited by 27
  4. 2022The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspectivecited by 24
  5. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
  6. 2023DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networkscited by 20
  7. 2023The Benefits of Mixup for Feature Learningcited by 19
  8. 2023Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Datacited by 18
  9. 2022Dimensionless machine learning: Imposing exact units equivariancecited by 16
  10. 2022Provably Learning Diverse Features in Multi-View Data with Midpoint Mixupcited by 14
  11. 2023On the Clean Generalization and Robust Overfitting in Adversarial Training from Two Theoretical Views: Representation Complexity and Training Dynamicscited by 5
  12. 2022Data-Efficient Augmentation for Training Neural Networkscited by 4
6 more
  1. 2023A Unified Framework for Generative Data Augmentation: A Comprehensive Surveycited by 4
  2. 2024Provable Benefit of Cutout and CutMix for Feature Learningcited by 4
  3. 2024For Better or For Worse? Learning Minimum Variance Features With Label Augmentationcited by 3
  4. 2023Deep Learning of Dynamical System Parameters from Return Maps as Imagescited by 0
  5. 2023Randomized Dimension Reduction with Statistical Guaranteescited by 0
  6. 2026The geometry of invariant learning: an information-theoretic analysis of data augmentation and generalizationcited by 0