# Boosting Multi-Label Image Classification with Complementary Parallel Self-Distillation

arXiv: https://arxiv.org/abs/2205.10986 · HTML: https://mlsanity.com/paper/2205.10986/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 23 May 2022.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [Billion-scale semi-supervised learning for image classification](https://mlsanity.com/paper/1905.00546/) (2019)
- [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016)
- [Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification](https://mlsanity.com/paper/2001.01536/) (2020)
- [Query2Label: A Simple Transformer Way to Multi-Label Classification](https://mlsanity.com/paper/2107.10834/) (2021)
- [General Multi-label Image Classification with Transformers](https://mlsanity.com/paper/2011.14027/) (2020)
- [Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets](https://mlsanity.com/paper/2007.09654/) (2020)
- [Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective](https://mlsanity.com/paper/2102.00650/) (2021)
- [Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition](https://mlsanity.com/paper/1908.07325/) (2019)
- [Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification](https://mlsanity.com/paper/1702.05891/) (2017)
- [Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognition](https://mlsanity.com/paper/2012.02994/) (2020)
- [Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification](https://mlsanity.com/paper/1912.07872/) (2019)
- [Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognition](https://mlsanity.com/paper/2007.01755/) (2020)
- [Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detection](https://mlsanity.com/paper/1809.05884/) (2018)
- [Deep Semantic Dictionary Learning for Multi-label Image Classification](https://mlsanity.com/paper/2012.12509/) (2020)

## Built on by (most cited first, 6 of 6)

- [Multi-Label Knowledge Distillation](https://mlsanity.com/paper/2308.06453/) (2023)
- [Semantic-Aware Dual Contrastive Learning for Multi-label Image Classification](https://mlsanity.com/paper/2307.09715/) (2023)
- [Counterfactual Reasoning for Multi-Label Image Classification via Patching-Based Training](https://mlsanity.com/paper/2404.06287/) (2024)
- [Correlative and Discriminative Label Grouping for Multi-Label Visual Prompt Tuning](https://mlsanity.com/paper/2504.09990/) (2025)
- [Occlusion-Robust FAU Recognition by Mining Latent Space of Masked Autoencoders](https://mlsanity.com/paper/2212.04029/) (2022)
- [Rethinking Multi-Label Image Classification With Deep Learning: Taxonomy, Challenge, and Outlook](https://mlsanity.com/paper/2607.00839/) (2026)

## Abstract (arXiv)

Multi-Label Image Classification (MLIC) approaches usually exploit label correlations to achieve good performance. However, emphasizing correlation like co-occurrence may overlook discriminative features of the target itself and lead to model overfitting, thus undermining the performance. In this study, we propose a generic framework named Parallel Self-Distillation (PSD) for boosting MLIC models. PSD decomposes the original MLIC task into several simpler MLIC sub-tasks via two elaborated complementary task decomposition strategies named Co-occurrence Graph Partition (CGP) and Dis-occurrence Graph Partition (DGP). Then, the MLIC models of fewer categories are trained with these sub-tasks in parallel for respectively learning the joint patterns and the category-specific patterns of labels. Finally, knowledge distillation is leveraged to learn a compact global ensemble of full categories with these learned patterns for reconciling the label correlation exploitation and model overfitting. Extensive results on MS-COCO and NUS-WIDE datasets demonstrate that our framework can be easily plugged into many MLIC approaches and improve performances of recent state-of-the-art approaches. The explainable visual study also further validates that our method is able to learn both the category-specific and co-occurring features. The source code is released at https://github.com/Robbie-Xu/CPSD.