# Generalization Gap in Data Augmentation: Insights from Illumination

arXiv: https://arxiv.org/abs/2404.07514 · HTML: https://mlsanity.com/paper/2404.07514/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 46 references that are in the map.
- First posted on arXiv 11 Apr 2024.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications](https://mlsanity.com/paper/2209.02976/) (2022)
- [Dataset Augmentation in Feature Space](https://mlsanity.com/paper/1702.05538/) (2017)
- [Domain Generalization for Medical Imaging Classification with Linear-Dependency Regularization](https://mlsanity.com/paper/2009.12829/) (2020)
- [Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification](https://mlsanity.com/paper/2012.00417/) (2020)
- [Data Augmentation using Random Image Cropping and Patching for Deep CNNs](https://mlsanity.com/paper/1811.09030/) (2018)
- [Regularizing Deep Networks with Semantic Data Augmentation](https://mlsanity.com/paper/2007.10538/) (2020)
- [Rain rendering for evaluating and improving robustness to bad weather](https://mlsanity.com/paper/2009.03683/) (2020)
- [Towards Recognizing Unseen Categories in Unseen Domains](https://mlsanity.com/paper/2007.12256/) (2020)
- [Combining Ensembles and Data Augmentation can Harm your Calibration](https://mlsanity.com/paper/2010.09875/) (2020)
- [Optimizing Deeper Transformers on Small Datasets](https://mlsanity.com/paper/2012.15355/) (2020)
- [Object Detection Under Rainy Conditions for Autonomous Vehicles: A Review of State-of-the-Art and Emerging Techniques](https://mlsanity.com/paper/2006.16471/) (2020)
- [When does Diversity Help Generalization in Classification Ensembles?](https://mlsanity.com/paper/1910.13631/) (2019)
- [Robust Place Categorization with Deep Domain Generalization](https://mlsanity.com/paper/1805.12048/) (2018)
- [Deep Image: Scaling up Image Recognition](https://mlsanity.com/paper/1501.02876/) (2015)

## Built on by (most cited first, 4 of 4)

- [The Impact of Semantic Pairs on Self-Supervised Representation Learning](https://mlsanity.com/paper/2510.08722/) (2025)
- [The Garbage Dataset (GD): A Multi-Class Image Benchmark for Automated Waste Segregation](https://mlsanity.com/paper/2602.10500/) (2026)
- [Lighting-Aware Representation Learning under Controllable Lighting Variation](https://mlsanity.com/paper/2606.06899/) (2026)
- [Transcript-Free Lightweight Detection of Alzheimer's Disease from Spontaneous Speech Using Handcrafted MFCC-Dominant Acoustic Biomarkers](https://mlsanity.com/paper/2607.10168/) (2026)

## Abstract (arXiv)

In the field of computer vision, data augmentation is widely used to enrich the feature complexity of training datasets with deep learning techniques. However, regarding the generalization capabilities of models, the difference in artificial features generated by data augmentation and natural visual features has not been fully revealed. This study introduces the concept of "visual representation variables" to define the possible visual variations in a task as a joint distribution of these variables. We focus on the visual representation variable "illumination", by simulating its distribution degradation and examining how data augmentation techniques enhance model performance on a classification task. Our goal is to investigate the differences in generalization between models trained with augmented data and those trained under real-world illumination conditions. Results indicate that after applying various data augmentation methods, model performance has significantly improved. Yet, a noticeable generalization gap still exists after utilizing various data augmentation methods, emphasizing the critical role of feature diversity in the training set for enhancing model generalization.