# Pure Noise to the Rescue of Insufficient Data: Improving Imbalanced Classification by Training on Random Noise Images

arXiv: https://arxiv.org/abs/2112.08810 · HTML: https://mlsanity.com/paper/2112.08810/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 65 references that are in the map.
- First posted on arXiv 16 Dec 2021.

## Builds on (most cited first, 30 of 38)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/1912.02413/) (2019)
- [Balanced Meta-Softmax for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2007.10740/) (2020)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Revisiting Batch Normalization For Practical Domain Adaptation](https://mlsanity.com/paper/1603.04779/) (2016)
- [Adversarial Examples Improve Image Recognition](https://mlsanity.com/paper/1911.09665/) (2019)
- [Improving Calibration for Long-Tailed Recognition](https://mlsanity.com/paper/2104.00466/) (2021)
- [CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2102.09559/) (2021)
- [Disentangling Label Distribution for Long-tailed Visual Recognition](https://mlsanity.com/paper/2012.00321/) (2020)
- [Feature Space Augmentation for Long-Tailed Data](https://mlsanity.com/paper/2008.03673/) (2020)
- [M2m: Imbalanced Classification via Major-to-minor Translation](https://mlsanity.com/paper/2004.00431/) (2020)
- [Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification](https://mlsanity.com/paper/2103.14267/) (2021)
- [Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspective](https://mlsanity.com/paper/2002.10826/) (2020)

## Built on by (most cited first, 16 of 16)

- [Feature Fusion from Head to Tail for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2306.06963/) (2023)
- [New Insights for the Stability-Plasticity Dilemma in Online Continual Learning](https://mlsanity.com/paper/2302.08741/) (2023)
- [Do we really have to filter out random noise in pre-training data for language models?](https://mlsanity.com/paper/2502.06604/) (2025)
- [A Method to Facilitate Membership Inference Attacks in Deep Learning Models](https://mlsanity.com/paper/2407.01919/) (2024)
- [Compound Batch Normalization for Long-tailed Image Classification](https://mlsanity.com/paper/2212.01007/) (2022)
- [Orthogonal Uncertainty Representation of Data Manifold for Robust Long-Tailed Learning](https://mlsanity.com/paper/2310.10090/) (2023)
- [Class-Aware Universum Inspired Re-Balance Learning for Long-Tailed Recognition](https://mlsanity.com/paper/2207.12808/) (2022)
- [Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach](https://mlsanity.com/paper/2403.00250/) (2024)
- [Differentially Private Prototypes for Imbalanced Transfer Learning](https://mlsanity.com/paper/2406.08039/) (2024)
- [Data-Centric Long-Tailed Image Recognition](https://mlsanity.com/paper/2311.01744/) (2023)
- [Text2Tree: Aligning Text Representation to the Label Tree Hierarchy for Imbalanced Medical Classification](https://mlsanity.com/paper/2311.16650/) (2023)
- [Class Instance Balanced Learning for Long-Tailed Classification](https://mlsanity.com/paper/2307.05322/) (2023)
- [Optimizing for ROC Curves on Class-Imbalanced Data by Training over a Family of Loss Functions](https://mlsanity.com/paper/2402.05400/) (2024)
- [Training Over a Distribution of Hyperparameters for Enhanced Performance and Adaptability on Imbalanced Classification](https://mlsanity.com/paper/2410.03588/) (2024)
- [DAMEL: Dual-Axis Multi-Expert Learning for Class-Imbalanced Learning](https://mlsanity.com/paper/2605.30135/) (2026)
- [Code-Poisoning Property Inference Attacks](https://mlsanity.com/paper/2607.15970/) (2026)

## Abstract (arXiv)

Despite remarkable progress on visual recognition tasks, deep neural-nets still struggle to generalize well when training data is scarce or highly imbalanced, rendering them extremely vulnerable to real-world examples. In this paper, we present a surprisingly simple yet highly effective method to mitigate this limitation: using pure noise images as additional training data. Unlike the common use of additive noise or adversarial noise for data augmentation, we propose an entirely different perspective by directly training on pure random noise images. We present a new Distribution-Aware Routing Batch Normalization layer (DAR-BN), which enables training on pure noise images in addition to natural images within the same network. This encourages generalization and suppresses overfitting. Our proposed method significantly improves imbalanced classification performance, obtaining state-of-the-art results on a large variety of long-tailed image classification datasets (CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT, Places-LT, and CelebA-5). Furthermore, our method is extremely simple and easy to use as a general new augmentation tool (on top of existing augmentations), and can be incorporated in any training scheme. It does not require any specialized data generation or training procedures, thus keeping training fast and efficient.