# ACE: Ally Complementary Experts for Solving Long-Tailed Recognition in One-Shot

arXiv: https://arxiv.org/abs/2108.02385 · HTML: https://mlsanity.com/paper/2108.02385/

## Key facts

- Cited by 86 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 5 Aug 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/1912.02413/) (2019)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Long-tailed Recognition by Routing Diverse Distribution-Aware Experts](https://mlsanity.com/paper/2010.01809/) (2020)
- [One weird trick for parallelizing convolutional neural networks](https://mlsanity.com/paper/1404.5997/) (2014)
- [Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification](https://mlsanity.com/paper/2001.01536/) (2020)
- [Remix: Rebalanced Mixup](https://mlsanity.com/paper/2007.03943/) (2020)
- [M2m: Imbalanced Classification via Major-to-minor Translation](https://mlsanity.com/paper/2004.00431/) (2020)
- [Range Loss for Deep Face Recognition with Long-tail](https://mlsanity.com/paper/1611.08976/) (2016)
- [The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation](https://mlsanity.com/paper/2007.11978/) (2020)
- [Equalization Loss v2: A New Gradient Balance Approach for Long-tailed Object Detection](https://mlsanity.com/paper/2012.08548/) (2020)
- [FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation](https://mlsanity.com/paper/2102.12867/) (2021)
- [Forest R-CNN: Large-Vocabulary Long-Tailed Object Detection and Instance Segmentation](https://mlsanity.com/paper/2008.05676/) (2020)
- [Towards Resolving the Challenge of Long-tail Distribution in UAV Images for Object Detection](https://mlsanity.com/paper/2011.03822/) (2020)
- [Imbalance-XGBoost: Leveraging Weighted and Focal Losses for Binary Label-Imbalanced Classification with XGBoost](https://mlsanity.com/paper/1908.01672/) (2019)

## Built on by (most cited first, 30 of 86)

- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Long-Tailed Recognition via Weight Balancing](https://mlsanity.com/paper/2203.14197/) (2022)
- [COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Prediction](https://mlsanity.com/paper/2312.01919/) (2023)
- [Nested Collaborative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2203.15359/) (2022)
- [Trustworthy Long-Tailed Classification](https://mlsanity.com/paper/2111.09030/) (2021)
- [TAM: Topology-Aware Margin Loss for Class-Imbalanced Node Classification](https://mlsanity.com/paper/2206.12917/) (2022)
- [Invariant Feature Learning for Generalized Long-Tailed Classification](https://mlsanity.com/paper/2207.09504/) (2022)
- [Learning Imbalanced Data with Vision Transformers](https://mlsanity.com/paper/2212.02015/) (2022)
- [ImbSAM: A Closer Look at Sharpness-Aware Minimization in Class-Imbalanced Recognition](https://mlsanity.com/paper/2308.07815/) (2023)
- [CUDA: Curriculum of Data Augmentation for Long-Tailed Recognition](https://mlsanity.com/paper/2302.05499/) (2023)
- [BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning](https://mlsanity.com/paper/2203.01522/) (2022)
- [A Systematic Review on Long-Tailed Learning](https://mlsanity.com/paper/2408.00483/) (2024)
- [Constructing Balance from Imbalance for Long-tailed Image Recognition](https://mlsanity.com/paper/2208.02567/) (2022)
- [A Unified Perspective for Loss-Oriented Imbalanced Learning via Localization](https://mlsanity.com/paper/2310.04752/) (2023)
- [Use Your Head: Improving Long-Tail Video Recognition](https://mlsanity.com/paper/2304.01143/) (2023)
- [Long-Tailed Visual Recognition via Self-Heterogeneous Integration with Knowledge Excavation](https://mlsanity.com/paper/2304.01279/) (2023)
- [Balanced Product of Calibrated Experts for Long-Tailed Recognition](https://mlsanity.com/paper/2206.05260/) (2022)
- [Feature Fusion from Head to Tail for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2306.06963/) (2023)
- [MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognition](https://mlsanity.com/paper/2308.09922/) (2023)
- [Adapting Multi-modal Large Language Model to Concept Drift From Pre-training Onwards](https://mlsanity.com/paper/2405.13459/) (2024)
- [PIDray: A Large-scale X-ray Benchmark for Real-World Prohibited Item Detection](https://mlsanity.com/paper/2211.10763/) (2022)
- [MoCaE: Mixture of Calibrated Experts Significantly Improves Object Detection](https://mlsanity.com/paper/2309.14976/) (2023)
- [Fed-GraB: Federated Long-tailed Learning with Self-Adjusting Gradient Balancer](https://mlsanity.com/paper/2310.07587/) (2023)
- [Long-tail Recognition via Compositional Knowledge Transfer](https://mlsanity.com/paper/2112.06741/) (2021)
- [DirMixE: Harnessing Test Agnostic Long-tail Recognition with Hierarchical Label Variations](https://mlsanity.com/paper/2405.07780/) (2024)
- [Improving Visual Prompt Tuning by Gaussian Neighborhood Minimization for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2410.21042/) (2024)
- [NCL++: Nested Collaborative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2306.16709/) (2023)
- [DeiT-LT Distillation Strikes Back for Vision Transformer Training on Long-Tailed Datasets](https://mlsanity.com/paper/2404.02900/) (2024)
- [Rethinking Semi-Supervised Imbalanced Node Classification from Bias-Variance Decomposition](https://mlsanity.com/paper/2310.18765/) (2023)
- [Three Heads Are Better Than One: Complementary Experts for Long-Tailed Semi-supervised Learning](https://mlsanity.com/paper/2312.15702/) (2023)

## Abstract (arXiv)

One-stage long-tailed recognition methods improve the overall performance in a "seesaw" manner, i.e., either sacrifice the head's accuracy for better tail classification or elevate the head's accuracy even higher but ignore the tail. Existing algorithms bypass such trade-off by a multi-stage training process: pre-training on imbalanced set and fine-tuning on balanced set. Though achieving promising performance, not only are they sensitive to the generalizability of the pre-trained model, but also not easily integrated into other computer vision tasks like detection and segmentation, where pre-training of classifiers solely is not applicable. In this paper, we propose a one-stage long-tailed recognition scheme, ally complementary experts (ACE), where the expert is the most knowledgeable specialist in a sub-set that dominates its training, and is complementary to other experts in the less-seen categories without being disturbed by what it has never seen. We design a distribution-adaptive optimizer to adjust the learning pace of each expert to avoid over-fitting. Without special bells and whistles, the vanilla ACE outperforms the current one-stage SOTA method by 3-10% on CIFAR10-LT, CIFAR100-LT, ImageNet-LT and iNaturalist datasets. It is also shown to be the first one to break the "seesaw" trade-off by improving the accuracy of the majority and minority categories simultaneously in only one stage. Code and trained models are at https://github.com/jrcai/ACE.