mlsanityData to 7 Oct 2026

Direct Differentiable Augmentation Search

arXiv 2104.04282 ↗First posted 9 Apr 2021cs.CV cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Data augmentation has been an indispensable tool to improve the performance of deep neural networks, however the augmentation can hardly transfer among different tasks and datasets. Consequently, a recent trend is to adopt AutoML technique to learn proper augmentation policy without extensive hand-crafted tuning. In this paper, we propose an efficient differentiable search algorithm called Direct Differentiable Augmentation Search (DDAS). It exploits meta-learning with one-step gradient update and continuous relaxation to the expected training loss for efficient search. Our DDAS can achieve efficient augmentation search without relying on approximations such as Gumbel Softmax or second order gradient approximation. To further reduce the adverse effect of improper augmentations, we organize the search space into a two level hierarchy, in which we first decide whether to apply augmentation, and then determine the specific augmentation policy. On standard image classification benchmarks, our DDAS achieves state-of-the-art performance and efficiency tradeoff while reducing the search cost dramatically, e.g. 0.15 GPU hours for CIFAR-10. In addition, we also use DDAS to search augmentation for object detection task and achieve comparable performance with AutoAugment, while being 1000x faster.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 43 of its 48 references that are in the map. First posted on arXiv 9 Apr 2021.

Builds on

43 of 48 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  5. 2016Wide Residual Networkscited by 3,057
  6. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  7. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  8. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  9. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  10. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  11. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  12. 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
31 more
  1. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  2. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  3. 2017Focal Loss for Dense Object Detectioncited by 749
  4. 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
  5. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  6. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 361
  7. 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
  8. 2018SNAS: Stochastic Neural Architecture Searchcited by 288
  9. 2019Adversarial Examples Improve Image Recognitioncited by 283
  10. 2019Fast AutoAugmentcited by 243
  11. 2019Population Based Augmentation: Efficient Learning of Augmentation Policy Schedulescited by 153
  12. 2019Learning Data Augmentation Strategies for Object Detectioncited by 139
  13. 2019Adversarial AutoAugmentcited by 111
  14. 2017Backpropagation through the Void: Optimizing control variates for black-box gradient estimationcited by 104
  15. 2018Data Augmentation by Pairing Samples for Images Classificationcited by 96
  16. 2019Faster AutoAugment: Learning Augmentation Strategies using Backpropagationcited by 96
  17. 2017Shake-Shake regularizationcited by 87
  18. 2020PointAugment: an Auto-Augmentation Framework for Point Cloud Classificationcited by 65
  19. 2020DADA: Differentiable Automatic Data Augmentationcited by 57
  20. 2020On the Generalization Effects of Linear Transformations in Data Augmentationcited by 47
  21. 2020Improving 3D Object Detection through Progressive Population Based Augmentationcited by 41
  22. 2019Online Hyper-parameter Learning for Auto-Augmentation Strategycited by 38
  23. 2019Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Datacited by 33
  24. 2020OnlineAugment: Online Data Augmentation with Less Domain Knowledgecited by 28
  25. 2020Improving Auto-Augment via Augmentation-Wise Weight Sharingcited by 28
  26. 2020Meta Approach to Data Augmentation Optimizationcited by 22
  27. 2020Learning Data Augmentation with Online Bilevel Optimization for Image Classificationcited by 20
  28. 2020MetaAugment: Sample-Aware Data Augmentation Policy Learningcited by 19
  29. 2019Stabilizing DARTS with Amended Gradient Estimation on Architectural Parameterscited by 11
  30. 2020Data augmentation with Mobius transformationscited by 9
  31. 2020Hypernetwork-Based Augmentationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 9; 2023: 4; 2024: 5; 2025: 5; 2026: 3.

28 papers.

Sort:

By area:

  1. 2024Learning to Transform Dynamically for Better Adversarial Transferabilitycited by 23
  2. 2022Deep AutoAugmentcited by 18
  3. 2022Automatic Data Augmentation via Invariance-Constrained Learningcited by 17
  4. 2024Automated data processing and feature engineering for deep learning and big data applications: a surveycited by 14
  5. 2024Relaxed Equivariance via Multitask Learningcited by 13
  6. 2022RangeAugment: Efficient Online Augmentation with Range Learningcited by 8
  7. 2023No Free Lunch in Self Supervised Representation Learningcited by 7
  8. 2023AROID: Improving Adversarial Robustness Through Online Instance-Wise Data Augmentationcited by 6
  9. 2023SLACK: Stable Learning of Augmentations with Cold-start and KL regularizationcited by 6
  10. 2022Universal Adaptive Data Augmentationcited by 5
  11. 2025Sample-aware RandAugment: Search-free Automatic Data Augmentation for Effective Image Recognitioncited by 5
  12. 2021DAAS: Differentiable Architecture and Augmentation Policy Searchcited by 4
16 more
  1. 2021DIVA: Dataset Derivative of a Learning Taskcited by 4
  2. 2022Empirical Evaluation of Data Augmentations for Biobehavioral Time Series Data with Deep Learningcited by 3
  3. 2024FreeAugment: Data Augmentation Search Across All Degrees of Freedomcited by 3
  4. 2022LEAVES: Learning Views for Time-Series Biobehavioral Data in Contrastive Learningcited by 2
  5. 2022Optimizing transformations for contrastive learning in a differentiable frameworkcited by 1
  6. 2022Learning Self-Regularized Adversarial Views for Self-Supervised Vision Transformerscited by 1
  7. 2024Data augmentation with automated machine learning: approaches and performance comparison with classical data augmentation methodscited by 1
  8. 2025Use as Many Surrogates as You Want: Selective Ensemble Attack to Unleash Transferability without Sacrificing Resource Efficiencycited by 1
  9. 2025Optimizing Data Augmentation through Bayesian Model Selectioncited by 1
  10. 2025Fine-grained Token Allocation Via Operation Pruning for Efficient MLLMscited by 1
  11. 2025On-the-Fly Data Augmentation via Gradient-Guided and Sample-Aware Influence Estimationcited by 1
  12. 2026Compositional Adversarial Training for Robust Visual Watermarkingcited by 1
  13. 2022SODA: Self-organizing data augmentation in deep neural networks -- Application to biomedical image segmentation taskscited by 0
  14. 2023Revisiting Data Augmentation in Model Compression: An Empirical and Comprehensive Studycited by 0
  15. 2026The geometry of invariant learning: an information-theoretic analysis of data augmentation and generalizationcited by 0
  16. 2026Ctrl-A: Control-Driven Online Data Augmentationcited by 0