# Online Hyper-parameter Learning for Auto-Augmentation Strategy

arXiv: https://arxiv.org/abs/1905.07373 · HTML: https://mlsanity.com/paper/1905.07373/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2019.

## Built on by (most cited first, 30 of 38)

- [TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentation](https://mlsanity.com/paper/2103.10158/) (2021)
- [Image Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2204.08610/) (2022)
- [LAS-AT: Adversarial Training with Learnable Attack Strategy](https://mlsanity.com/paper/2203.06616/) (2022)
- [Vision Transformer with Quadrangle Attention](https://mlsanity.com/paper/2303.15105/) (2023)
- [VSA: Learning Varied-Size Window Attention in Vision Transformers](https://mlsanity.com/paper/2204.08446/) (2022)
- [HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks](https://mlsanity.com/paper/2102.02515/) (2021)
- [TeachAugment: Data Augmentation Optimization Using Teacher Knowledge](https://mlsanity.com/paper/2202.12513/) (2022)
- [Direct Differentiable Augmentation Search](https://mlsanity.com/paper/2104.04282/) (2021)
- [Swift Sampler: Efficient Learning of Sampler by 10 Parameters](https://mlsanity.com/paper/2410.05578/) (2024)
- [CADDA: Class-wise Automatic Differentiable Data Augmentation for EEG Signals](https://mlsanity.com/paper/2106.13695/) (2021)
- [AutoDO: Robust AutoAugment for Biased Data with Label Noise via Scalable Probabilistic Implicit Differentiation](https://mlsanity.com/paper/2103.05863/) (2021)
- [Automated data processing and feature engineering for deep learning and big data applications: a survey](https://mlsanity.com/paper/2403.11395/) (2024)
- [Learning Augmentation Distributions using Transformed Risk Minimization](https://mlsanity.com/paper/2111.08190/) (2021)
- [When to Learn What: Model-Adaptive Data Augmentation Curriculum](https://mlsanity.com/paper/2309.04747/) (2023)
- [A Survey of Automated Data Augmentation Algorithms for Deep Learning-based Image Classification Tasks](https://mlsanity.com/paper/2206.06544/) (2022)
- [SelectAugment: Hierarchical Deterministic Sample Selection for Data Augmentation](https://mlsanity.com/paper/2112.02862/) (2021)
- [Local Patch AutoAugment with Multi-Agent Collaboration](https://mlsanity.com/paper/2103.11099/) (2021)
- [DHA: End-to-End Joint Optimization of Data Augmentation Policy, Hyper-parameter and Architecture](https://mlsanity.com/paper/2109.05765/) (2021)
- [AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentation](https://mlsanity.com/paper/2306.07197/) (2023)
- [SLACK: Stable Learning of Augmentations with Cold-start and KL regularization](https://mlsanity.com/paper/2306.09998/) (2023)
- [Anchor Data Augmentation](https://mlsanity.com/paper/2311.06965/) (2023)
- [Universal Adaptive Data Augmentation](https://mlsanity.com/paper/2207.06658/) (2022)
- [Sample-aware RandAugment: Search-free Automatic Data Augmentation for Effective Image Recognition](https://mlsanity.com/paper/2508.08004/) (2025)
- [FreeAugment: Data Augmentation Search Across All Degrees of Freedom](https://mlsanity.com/paper/2409.04820/) (2024)
- [Adversarial Learning of Hard Positives for Place Recognition](https://mlsanity.com/paper/2205.03871/) (2022)
- [ParticleAugment: Sampling-Based Data Augmentation](https://mlsanity.com/paper/2106.08693/) (2021)
- [AutoAdapt: Automated Segmentation Network Search for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2106.13227/) (2021)
- [Automatic Data Augmentation Learning using Bilevel Optimization for Histopathological Images](https://mlsanity.com/paper/2307.11808/) (2023)
- [Data augmentation with automated machine learning: approaches and performance comparison with classical data augmentation methods](https://mlsanity.com/paper/2403.08352/) (2024)
- [Beyond Benchmarks of IUGC: Rethinking Requirements of Deep Learning Methods for Intrapartum Ultrasound Biometry from Fetal Ultrasound Videos](https://mlsanity.com/paper/2602.12922/) (2026)

## Abstract (arXiv)

Data augmentation is critical to the success of modern deep learning techniques. In this paper, we propose Online Hyper-parameter Learning for Auto-Augmentation (OHL-Auto-Aug), an economical solution that learns the augmentation policy distribution along with network training. Unlike previous methods on auto-augmentation that search augmentation strategies in an offline manner, our method formulates the augmentation policy as a parameterized probability distribution, thus allowing its parameters to be optimized jointly with network parameters. Our proposed OHL-Auto-Aug eliminates the need of re-training and dramatically reduces the cost of the overall search process, while establishes significantly accuracy improvements over baseline models. On both CIFAR-10 and ImageNet, our method achieves remarkable on search accuracy, 60x faster on CIFAR-10 and 24x faster on ImageNet, while maintaining competitive accuracies.