# IMB-NAS: Neural Architecture Search for Imbalanced Datasets

arXiv: https://arxiv.org/abs/2210.00136 · HTML: https://mlsanity.com/paper/2210.00136/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 30 Sep 2022.

## Builds on (most cited first, 24 of 24)

- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Single Path One-Shot Neural Architecture Search with Uniform Sampling](https://mlsanity.com/paper/1904.00420/) (2019)
- [BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/1912.02413/) (2019)
- [Large-Scale Evolution of Image Classifiers](https://mlsanity.com/paper/1703.01041/) (2017)
- [Designing Neural Network Architectures using Reinforcement Learning](https://mlsanity.com/paper/1611.02167/) (2016)
- [Distribution Alignment: A Unified Framework for Long-tail Visual Recognition](https://mlsanity.com/paper/2103.16370/) (2021)
- [Feature Space Augmentation for Long-Tailed Data](https://mlsanity.com/paper/2008.03673/) (2020)
- [NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size](https://mlsanity.com/paper/2009.00437/) (2020)
- [Neural Architecture Transfer](https://mlsanity.com/paper/2005.05859/) (2020)
- [Posterior Re-calibration for Imbalanced Datasets](https://mlsanity.com/paper/2010.11820/) (2020)
- [Solving Long-tailed Recognition with Deep Realistic Taxonomic Classifier](https://mlsanity.com/paper/2007.09898/) (2020)
- [Compatibility-aware Heterogeneous Visual Search](https://mlsanity.com/paper/2105.06047/) (2021)
- [ELF: An Early-Exiting Framework for Long-Tailed Classification](https://mlsanity.com/paper/2006.11979/) (2020)
- [NASTransfer: Analyzing Architecture Transferability in Large Scale Neural Architecture Search](https://mlsanity.com/paper/2006.13314/) (2020)

## Built on by (most cited first, 2 of 2)

- [An Efficient NAS-based Approach for Handling Imbalanced Datasets](https://mlsanity.com/paper/2406.16972/) (2024)
- [Fair Differentiable Neural Network Architecture Search for Long-Tailed Data with Self-Supervised Learning](https://mlsanity.com/paper/2406.16949/) (2024)

## Abstract (arXiv)

Class imbalance is a ubiquitous phenomenon occurring in real world data distributions. To overcome its detrimental effect on training accurate classifiers, existing work follows three major directions: class re-balancing, information transfer, and representation learning. In this paper, we propose a new and complementary direction for improving performance on long tailed datasets - optimizing the backbone architecture through neural architecture search (NAS). We find that an architecture's accuracy obtained on a balanced dataset is not indicative of good performance on imbalanced ones. This poses the need for a full NAS run on long tailed datasets which can quickly become prohibitively compute intensive. To alleviate this compute burden, we aim to efficiently adapt a NAS super-network from a balanced source dataset to an imbalanced target one. Among several adaptation strategies, we find that the most effective one is to retrain the linear classification head with reweighted loss, while freezing the backbone NAS super-network trained on a balanced source dataset. We perform extensive experiments on multiple datasets and provide concrete insights to optimize architectures for long tailed datasets.