# Faster AutoAugment: Learning Augmentation Strategies using Backpropagation

arXiv: https://arxiv.org/abs/1911.06987 · HTML: https://mlsanity.com/paper/1911.06987/

## Key facts

- Cited by 96 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2019.

## Built on by (most cited first, 30 of 96)

- [Contrastive Learning with Stronger Augmentations](https://mlsanity.com/paper/2104.07713/) (2021)
- [A Comprehensive Survey of Image Augmentation Techniques for Deep Learning](https://mlsanity.com/paper/2205.01491/) (2022)
- [Adversarial Masking for Self-Supervised Learning](https://mlsanity.com/paper/2201.13100/) (2022)
- [AutoGCL: Automated Graph Contrastive Learning via Learnable View Generators](https://mlsanity.com/paper/2109.10259/) (2021)
- [Crafting Better Contrastive Views for Siamese Representation Learning](https://mlsanity.com/paper/2202.03278/) (2022)
- [Time Series Contrastive Learning with Information-Aware Augmentations](https://mlsanity.com/paper/2303.11911/) (2023)
- [AutoBalance: Optimized Loss Functions for Imbalanced Data](https://mlsanity.com/paper/2201.01212/) (2022)
- [Learning Partial Equivariances from Data](https://mlsanity.com/paper/2110.10211/) (2021)
- [TeachAugment: Data Augmentation Optimization Using Teacher Knowledge](https://mlsanity.com/paper/2202.12513/) (2022)
- [On Implicit Bias in Overparameterized Bilevel Optimization](https://mlsanity.com/paper/2212.14032/) (2022)
- [CUDA: Curriculum of Data Augmentation for Long-Tailed Recognition](https://mlsanity.com/paper/2302.05499/) (2023)
- [Direct Differentiable Augmentation Search](https://mlsanity.com/paper/2104.04282/) (2021)
- [Image Data Augmentation Approaches: A Comprehensive Survey and Future directions](https://mlsanity.com/paper/2301.02830/) (2023)
- [Understanding Test-Time Augmentation](https://mlsanity.com/paper/2402.06892/) (2024)
- [CADDA: Class-wise Automatic Differentiable Data Augmentation for EEG Signals](https://mlsanity.com/paper/2106.13695/) (2021)
- [Object-Aware Cropping for Self-Supervised Learning](https://mlsanity.com/paper/2112.00319/) (2021)
- [Automated Data Augmentations for Graph Classification](https://mlsanity.com/paper/2202.13248/) (2022)
- [Deep AutoAugment](https://mlsanity.com/paper/2203.06172/) (2022)
- [Automatic Data Augmentation via Invariance-Constrained Learning](https://mlsanity.com/paper/2209.15031/) (2022)
- [Augmentation-Free Graph Contrastive Learning with Performance Guarantee](https://mlsanity.com/paper/2204.04874/) (2022)
- [DivAug: Plug-in Automated Data Augmentation with Explicit Diversity Maximization](https://mlsanity.com/paper/2103.14545/) (2021)
- [Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation](https://mlsanity.com/paper/2107.11252/) (2021)
- [Text AutoAugment: Learning Compositional Augmentation Policy for Text Classification](https://mlsanity.com/paper/2109.00523/) (2021)
- [Learning Instance-Specific Augmentations by Capturing Local Invariances](https://mlsanity.com/paper/2206.00051/) (2022)
- [AutoDO: Robust AutoAugment for Biased Data with Label Noise via Scalable Probabilistic Implicit Differentiation](https://mlsanity.com/paper/2103.05863/) (2021)
- [An overview of mixing augmentation methods and augmentation strategies](https://mlsanity.com/paper/2107.09887/) (2021)
- [Learning Graph Augmentations to Learn Graph Representations](https://mlsanity.com/paper/2201.09830/) (2022)
- [Automated data processing and feature engineering for deep learning and big data applications: a survey](https://mlsanity.com/paper/2403.11395/) (2024)
- [Learning Augmentation Distributions using Transformed Risk Minimization](https://mlsanity.com/paper/2111.08190/) (2021)
- [Masked Autoencoders are Robust Data Augmentors](https://mlsanity.com/paper/2206.04846/) (2022)

## Abstract (arXiv)

Data augmentation methods are indispensable heuristics to boost the performance of deep neural networks, especially in image recognition tasks. Recently, several studies have shown that augmentation strategies found by search algorithms outperform hand-made strategies. Such methods employ black-box search algorithms over image transformations with continuous or discrete parameters and require a long time to obtain better strategies. In this paper, we propose a differentiable policy search pipeline for data augmentation, which is much faster than previous methods. We introduce approximate gradients for several transformation operations with discrete parameters as well as the differentiable mechanism for selecting operations. As the objective of training, we minimize the distance between the distributions of augmented data and the original data, which can be differentiated. We show that our method, Faster AutoAugment, achieves significantly faster searching than prior work without a performance drop.