# Milking CowMask for Semi-Supervised Image Classification

arXiv: https://arxiv.org/abs/2003.12022 · HTML: https://mlsanity.com/paper/2003.12022/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2020.

## Built on by (most cited first, 30 of 35)

- [Self-supervised Pretraining of Visual Features in the Wild](https://mlsanity.com/paper/2103.01988/) (2021)
- [Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization](https://mlsanity.com/paper/2104.05833/) (2021)
- [Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision](https://mlsanity.com/paper/2202.08360/) (2022)
- [Context-Aware Mixup for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2108.03557/) (2021)
- [Contrast to Divide: Self-Supervised Pre-Training for Learning with Noisy Labels](https://mlsanity.com/paper/2103.13646/) (2021)
- [SVFormer: Semi-supervised Video Transformer for Action Recognition](https://mlsanity.com/paper/2211.13222/) (2022)
- [Semi-supervised Vision Transformers at Scale](https://mlsanity.com/paper/2208.05688/) (2022)
- [A Comprehensive Survey on Data Augmentation](https://mlsanity.com/paper/2405.09591/) (2024)
- [Revisiting Consistency Regularization for Semi-Supervised Learning](https://mlsanity.com/paper/2112.05825/) (2021)
- [MixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworks](https://mlsanity.com/paper/2103.06132/) (2021)
- [All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-training](https://mlsanity.com/paper/2104.05248/) (2021)
- [Deliberated Domain Bridging for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2209.07695/) (2022)
- [A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainability](https://mlsanity.com/paper/2212.10888/) (2022)
- [PIT: Position-Invariant Transform for Cross-FoV Domain Adaptation](https://mlsanity.com/paper/2108.07142/) (2021)
- [DytanVO: Joint Refinement of Visual Odometry and Motion Segmentation in Dynamic Environments](https://mlsanity.com/paper/2209.08430/) (2022)
- [Survey: Image Mixing and Deleting for Data Augmentation](https://mlsanity.com/paper/2106.07085/) (2021)
- [Semi-Supervised Vision Transformers](https://mlsanity.com/paper/2111.11067/) (2021)
- [Imposing Consistency for Optical Flow Estimation](https://mlsanity.com/paper/2204.07262/) (2022)
- [A Survey on Mixup Augmentations and Beyond](https://mlsanity.com/paper/2409.05202/) (2024)
- [Open-Set Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2405.19899/) (2024)
- [RoPAWS: Robust Semi-supervised Representation Learning from Uncurated Data](https://mlsanity.com/paper/2302.14483/) (2023)
- [ML-BPM: Multi-teacher Learning with Bidirectional Photometric Mixing for Open Compound Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2207.09045/) (2022)
- [Optical Flow for Autonomous Driving: Applications, Challenges and Improvements](https://mlsanity.com/paper/2301.04422/) (2023)
- [Mask-based Data Augmentation for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2101.10156/) (2021)
- [S5Mars: Semi-Supervised Learning for Mars Semantic Segmentation](https://mlsanity.com/paper/2207.01200/) (2022)
- [Towards Unsupervised Domain Bridging via Image Degradation in Semantic Segmentation](https://mlsanity.com/paper/2412.10339/) (2024)
- [Pseudo-Label Noise Suppression Techniques for Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2210.10426/) (2022)
- [Lidar-based Norwegian tree species detection using deep learning](https://mlsanity.com/paper/2311.06066/) (2023)
- [Extending Temporal Data Augmentation for Video Action Recognition](https://mlsanity.com/paper/2211.04888/) (2022)
- [SCMix: Stochastic Compound Mixing for Open Compound Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2405.14278/) (2024)

## Abstract (arXiv)

Consistency regularization is a technique for semi-supervised learning that underlies a number of strong results for classification with few labeled data. It works by encouraging a learned model to be robust to perturbations on unlabeled data. Here, we present a novel mask-based augmentation method called CowMask. Using it to provide perturbations for semi-supervised consistency regularization, we achieve a state-of-the-art result on ImageNet with 10% labeled data, with a top-5 error of 8.76% and top-1 error of 26.06%. Moreover, we do so with a method that is much simpler than many alternatives. We further investigate the behavior of CowMask for semi-supervised learning by running many smaller scale experiments on the SVHN, CIFAR-10 and CIFAR-100 data sets, where we achieve results competitive with the state of the art, indicating that CowMask is widely applicable. We open source our code at https://github.com/google-research/google-research/tree/master/milking_cowmask