# TeachAugment: Data Augmentation Optimization Using Teacher Knowledge

arXiv: https://arxiv.org/abs/2202.12513 · HTML: https://mlsanity.com/paper/2202.12513/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 66 references that are in the map.
- First posted on arXiv 25 Feb 2022.

## Builds on (most cited first, 30 of 49)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Fast AutoAugment](https://mlsanity.com/paper/1905.00397/) (2019)
- [TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentation](https://mlsanity.com/paper/2103.10158/) (2021)
- [Deep Pyramidal Residual Networks](https://mlsanity.com/paper/1610.02915/) (2016)

## Built on by (most cited first, 30 of 37)

- [Learning Multimodal Data Augmentation in Feature Space](https://mlsanity.com/paper/2212.14453/) (2022)
- [Fourier-basis Functions to Bridge Augmentation Gap: Rethinking Frequency Augmentation in Image Classification](https://mlsanity.com/paper/2403.01944/) (2024)
- [When to Learn What: Model-Adaptive Data Augmentation Curriculum](https://mlsanity.com/paper/2309.04747/) (2023)
- [EntAugment: Entropy-Driven Adaptive Data Augmentation Framework for Image Classification](https://mlsanity.com/paper/2409.06290/) (2024)
- [SUMix: Mixup with Semantic and Uncertain Information](https://mlsanity.com/paper/2407.07805/) (2024)
- [Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning](https://mlsanity.com/paper/2305.16379/) (2023)
- [Improving the Transferability of Adversarial Attacks on Face Recognition with Beneficial Perturbation Feature Augmentation](https://mlsanity.com/paper/2210.16117/) (2022)
- [AdaAugment: A Tuning-Free and Adaptive Approach to Enhance Data Augmentation](https://mlsanity.com/paper/2405.11467/) (2024)
- [SLACK: Stable Learning of Augmentations with Cold-start and KL regularization](https://mlsanity.com/paper/2306.09998/) (2023)
- [Deeper Insights into the Robustness of ViTs towards Common Corruptions](https://mlsanity.com/paper/2204.12143/) (2022)
- [Teaching What You Should Teach: A Data-Based Distillation Method](https://mlsanity.com/paper/2212.05422/) (2022)
- [Domain Generalization by Rejecting Extreme Augmentations](https://mlsanity.com/paper/2310.06670/) (2023)
- [Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clients](https://mlsanity.com/paper/2503.09206/) (2025)
- [Sample-aware RandAugment: Search-free Automatic Data Augmentation for Effective Image Recognition](https://mlsanity.com/paper/2508.08004/) (2025)
- [Improving the Transferability of Adversarial Attacks on Face Recognition with Diverse Parameters Augmentation](https://mlsanity.com/paper/2411.15555/) (2024)
- [ReSmooth: Detecting and Utilizing OOD Samples when Training with Data Augmentation](https://mlsanity.com/paper/2205.12606/) (2022)
- [Few-Shot Learning from Augmented Label-Uncertain Queries in Bongard-HOI](https://mlsanity.com/paper/2312.10586/) (2023)
- [FreeAugment: Data Augmentation Search Across All Degrees of Freedom](https://mlsanity.com/paper/2409.04820/) (2024)
- [SCOUT: Semi-supervised Camouflaged Object Detection by Utilizing Text and Adaptive Data Selection](https://mlsanity.com/paper/2508.17843/) (2025)
- [Self-Guided Masked Autoencoders for Domain-Agnostic Self-Supervised Learning](https://mlsanity.com/paper/2402.14789/) (2024)
- [Long-Tailed Backdoor Attack Using Dynamic Data Augmentation Operations](https://mlsanity.com/paper/2410.12955/) (2024)
- [Learning Self-Regularized Adversarial Views for Self-Supervised Vision Transformers](https://mlsanity.com/paper/2210.08458/) (2022)
- [Clothes Grasping and Unfolding Based on RGB-D Semantic Segmentation](https://mlsanity.com/paper/2305.03259/) (2023)
- [Target-driven One-Shot Unsupervised Domain Adaptation](https://mlsanity.com/paper/2305.04628/) (2023)
- [Learnable Data Augmentation for One-Shot Unsupervised Domain Adaptation](https://mlsanity.com/paper/2310.02201/) (2023)
- [DualAug: Exploiting Additional Heavy Augmentation with OOD Data Rejection](https://mlsanity.com/paper/2310.08139/) (2023)
- [Data augmentation with automated machine learning: approaches and performance comparison with classical data augmentation methods](https://mlsanity.com/paper/2403.08352/) (2024)
- [Bidirectional Multi-Step Domain Generalization for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2403.10782/) (2024)
- [CAAP: Class-Dependent Automatic Data Augmentation Based On Adaptive Policies For Time Series](https://mlsanity.com/paper/2404.00898/) (2024)
- [Enhancing Small-Scale Dataset Expansion with Triplet-Connection-based Sample Re-Weighting](https://mlsanity.com/paper/2508.07723/) (2025)

## Abstract (arXiv)

Optimization of image transformation functions for the purpose of data augmentation has been intensively studied. In particular, adversarial data augmentation strategies, which search augmentation maximizing task loss, show significant improvement in the model generalization for many tasks. However, the existing methods require careful parameter tuning to avoid excessively strong deformations that take away image features critical for acquiring generalization. In this paper, we propose a data augmentation optimization method based on the adversarial strategy called TeachAugment, which can produce informative transformed images to the model without requiring careful tuning by leveraging a teacher model. Specifically, the augmentation is searched so that augmented images are adversarial for the target model and recognizable for the teacher model. We also propose data augmentation using neural networks, which simplifies the search space design and allows for updating of the data augmentation using the gradient method. We show that TeachAugment outperforms existing methods in experiments of image classification, semantic segmentation, and unsupervised representation learning tasks.