mlsanityData to 7 Oct 2026

AutoDO: Robust AutoAugment for Biased Data with Label Noise via Scalable Probabilistic Implicit Differentiation

arXiv 2103.05863 ↗First posted 10 Mar 2021cs.CV cs.AI cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

AutoAugment has sparked an interest in automated augmentation methods for deep learning models. These methods estimate image transformation policies for train data that improve generalization to test data. While recent papers evolved in the direction of decreasing policy search complexity, we show that those methods are not robust when applied to biased and noisy data. To overcome these limitations, we reformulate AutoAugment as a generalized automated dataset optimization (AutoDO) task that minimizes the distribution shift between test data and distorted train dataset. In our AutoDO model, we explicitly estimate a set of per-point hyperparameters to flexibly change distribution of train data. In particular, we include hyperparameters for augmentation, loss weights, and soft-labels that are jointly estimated using implicit differentiation. We develop a theoretical probabilistic interpretation of this framework using Fisher information and show that its complexity scales linearly with the dataset size. Our experiments on SVHN, CIFAR-10/100, and ImageNet classification show up to 9.3% improvement for biased datasets with label noise compared to prior methods and, importantly, up to 36.6% gain for underrepresented SVHN classes.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 33 of its 49 references that are in the map. First posted on arXiv 10 Mar 2021.

Builds on

33 of 49 references.

References · 33

  1. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  3. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  4. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  5. 2016Wide Residual Networkscited by 3,057
  6. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  7. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  8. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  9. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  10. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  11. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  12. 2017Random Erasing Data Augmentationcited by 1,235
21 more
  1. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  2. 2019Deep Equilibrium Modelscited by 603
  3. 2019Meta-Learning with Implicit Gradientscited by 523
  4. 2017Population Based Training of Neural Networkscited by 351
  5. 2019Optimizing Millions of Hyperparameters by Implicit Differentiationcited by 303
  6. 2018Joint Optimization Framework for Learning with Noisy Labelscited by 293
  7. 2019Fast AutoAugmentcited by 243
  8. 2016Hyperparameter optimization with approximate gradientcited by 231
  9. 2019Kornia: an Open Source Differentiable Computer Vision Library for PyTorchcited by 231
  10. 2019Probabilistic End-to-end Noise Correction for Learning with Noisy Labelscited by 210
  11. 2019Population Based Augmentation: Efficient Learning of Augmentation Policy Schedulescited by 153
  12. 2019Adversarial AutoAugmentcited by 111
  13. 2019Faster AutoAugment: Learning Augmentation Strategies using Backpropagationcited by 96
  14. 2021A Comprehensive Study on Face Recognition Biases Beyond Demographicscited by 86
  15. 2017Continuously Differentiable Exponential Linear Unitscited by 73
  16. 2020DADA: Differentiable Automatic Data Augmentationcited by 57
  17. 2020Every Model Learned by Gradient Descent Is Approximately a Kernel Machinecited by 53
  18. 2017A Bayesian Data Augmentation Approach for Learning Deep Modelscited by 50
  19. 2019Online Hyper-parameter Learning for Auto-Augmentation Strategycited by 38
  20. 2020UniformAugment: A Search-free Probabilistic Data Augmentation Approachcited by 38
  21. 2020Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervisioncited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 3; 2024: 4; 2025: 1; 2026: 1.

14 papers.

Sort:

  1. 2023CUDA: Curriculum of Data Augmentation for Long-Tailed Recognitioncited by 30
  2. 2023Making Scalable Meta Learning Practicalcited by 17
  3. 2022A Survey of Automated Data Augmentation Algorithms for Deep Learning-based Image Classification Taskscited by 12
  4. 2024FedFixer: Mitigating Heterogeneous Label Noise in Federated Learningcited by 8
  5. 2022ReSmooth: Detecting and Utilizing OOD Samples when Training with Data Augmentationcited by 3
  6. 2024Towards Trustworthy Machine Learning in Production: An Overview of the Robustness in MLOps Approachcited by 3
  7. 2022Combating Noisy Labels in Long-Tailed Image Classificationcited by 2
  8. 2025Hyperbolic Dual Feature Augmentation for Open-Environmentcited by 2
  9. 2023DualAug: Exploiting Additional Heavy Augmentation with OOD Data Rejectioncited by 1
  10. 2024Data augmentation with automated machine learning: approaches and performance comparison with classical data augmentation methodscited by 1
  11. 2024Fisher-aware Quantization for DETR Detectors with Critical-category Objectivescited by 1
  12. 2022Efficient Scheduling of Data Augmentation for Deep Reinforcement Learningcited by 0
2 more
  1. 2022A Study of Deep CNN Model with Labeling Noise Based on Granular-ball Computingcited by 0
  2. 2026Variational Rectification Inference for Learning with Noisy Labelscited by 0