# Semantic-Aware Representation Blending for Multi-Label Image Recognition with Partial Labels

arXiv: https://arxiv.org/abs/2203.02172 · HTML: https://mlsanity.com/paper/2203.02172/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 4 Mar 2022.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup](https://mlsanity.com/paper/2009.06962/) (2020)
- [Learning Visual Features from Large Weakly Supervised Data](https://mlsanity.com/paper/1511.02251/) (2015)
- [Knowledge-Embedded Routing Network for Scene Graph Generation](https://mlsanity.com/paper/1903.03326/) (2019)
- [Learning a Deep ConvNet for Multi-label Classification with Partial Labels](https://mlsanity.com/paper/1902.09720/) (2019)
- [Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition](https://mlsanity.com/paper/1908.07325/) (2019)
- [Multi-label Image Recognition by Recurrently Discovering Attentional Regions](https://mlsanity.com/paper/1711.02816/) (2017)
- [Computing Graph Neural Networks: A Survey from Algorithms to Accelerators](https://mlsanity.com/paper/2010.00130/) (2020)
- [Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognition](https://mlsanity.com/paper/2012.02994/) (2020)
- [Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition](https://mlsanity.com/paper/2009.09450/) (2020)
- [CNN: Single-label to Multi-label](https://mlsanity.com/paper/1406.5726/) (2014)
- [HSVA: Hierarchical Semantic-Visual Adaptation for Zero-Shot Learning](https://mlsanity.com/paper/2109.15163/) (2021)
- [Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognition](https://mlsanity.com/paper/2007.01755/) (2020)
- [Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning](https://mlsanity.com/paper/2008.00923/) (2020)
- [Recurrent Attentional Reinforcement Learning for Multi-label Image Recognition](https://mlsanity.com/paper/1712.07465/) (2017)
- [Structured Semantic Transfer for Multi-Label Recognition with Partial Labels](https://mlsanity.com/paper/2112.10941/) (2021)
- [Mixup-CAM: Weakly-supervised Semantic Segmentation via Uncertainty Regularization](https://mlsanity.com/paper/2008.01201/) (2020)
- [Knowledge Graph Transfer Network for Few-Shot Recognition](https://mlsanity.com/paper/1911.09579/) (2019)
- [Knowledge-Embedded Representation Learning for Fine-Grained Image Recognition](https://mlsanity.com/paper/1807.00505/) (2018)
- [Relational Graph Learning for Grounded Video Description Generation](https://mlsanity.com/paper/2112.00967/) (2021)
- [Instance-weighted Central Similarity for Multi-label Image Retrieval](https://mlsanity.com/paper/2108.05274/) (2021)

## Built on by (most cited first, 29 of 29)

- [DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2206.09541/) (2022)
- [Texts as Images in Prompt Tuning for Multi-Label Image Recognition](https://mlsanity.com/paper/2211.12739/) (2022)
- [CDUL: CLIP-Driven Unsupervised Learning for Multi-Label Image Classification](https://mlsanity.com/paper/2307.16634/) (2023)
- [DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2308.01890/) (2023)
- [Heterogeneous Semantic Transfer for Multi-label Recognition with Partial Labels](https://mlsanity.com/paper/2205.11131/) (2022)
- [TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP Without Training](https://mlsanity.com/paper/2312.12828/) (2023)
- [TAI++: Text as Image for Multi-Label Image Classification by Co-Learning Transferable Prompt](https://mlsanity.com/paper/2405.06926/) (2024)
- [Spatial-Temporal Knowledge-Embedded Transformer for Video Scene Graph Generation](https://mlsanity.com/paper/2309.13237/) (2023)
- [Text-Region Matching for Multi-Label Image Recognition with Missing Labels](https://mlsanity.com/paper/2407.18520/) (2024)
- [Data-free Multi-label Image Recognition via LLM-powered Prompt Tuning](https://mlsanity.com/paper/2403.01209/) (2024)
- [Semantic-Aware Graph Matching Mechanism for Multi-Label Image Recognition](https://mlsanity.com/paper/2304.11275/) (2023)
- [Positive Label Is All You Need for Multi-Label Classification](https://mlsanity.com/paper/2306.16016/) (2023)
- [Semantic Representation and Dependency Learning for Multi-Label Image Recognition](https://mlsanity.com/paper/2204.03795/) (2022)
- [Dual-Perspective Semantic-Aware Representation Blending for Multi-Label Image Recognition with Partial Labels](https://mlsanity.com/paper/2205.13092/) (2022)
- [Semantic Contrastive Bootstrapping for Single-positive Multi-label Recognition](https://mlsanity.com/paper/2307.07680/) (2023)
- [Text as Image: Learning Transferable Adapter for Multi-Label Classification](https://mlsanity.com/paper/2312.04160/) (2023)
- [Classifier-guided CLIP Distillation for Unsupervised Multi-label Classification](https://mlsanity.com/paper/2503.16873/) (2025)
- [Category-wise Fine-Tuning: Resisting Incorrect Pseudo-Labels in Multi-Label Image Classification with Partial Labels](https://mlsanity.com/paper/2401.16991/) (2024)
- [Rethinking Prompting Strategies for Multi-Label Recognition with Partial Annotations](https://mlsanity.com/paper/2409.08381/) (2024)
- [Learning Semantic-Aware Representation in Visual-Language Models for Multi-Label Recognition with Partial Labels](https://mlsanity.com/paper/2412.10843/) (2024)
- [Text to Image for Multi-Label Image Recognition with Joint Prompt-Adapter Learning](https://mlsanity.com/paper/2506.10575/) (2025)
- [Exploring Partial Multi-Label Learning via Integrating Semantic Co-occurrence Knowledge](https://mlsanity.com/paper/2507.05992/) (2025)
- [PSScreen: Partially Supervised Multiple Retinal Disease Screening](https://mlsanity.com/paper/2508.10549/) (2025)
- [Category-Adaptive Label Discovery and Noise Rejection for Multi-label Image Recognition with Partial Positive Labels](https://mlsanity.com/paper/2211.07846/) (2022)
- [Free Performance Gain from Mixing Multiple Partially Labeled Samples in Multi-label Image Classification](https://mlsanity.com/paper/2405.15860/) (2024)
- [Incomplete Multi-Label Image Recognition by Co-learning Semantic-Aware Features and Label Recovery](https://mlsanity.com/paper/2510.10055/) (2025)
- [PSScreen V2: Partially Supervised Multiple Retinal Disease Screening](https://mlsanity.com/paper/2510.22589/) (2025)
- [[CLS] is Not Enough: Multi-Label Recognition via Patch-Level Inference and Adaptive Aggregation](https://mlsanity.com/paper/2605.25821/) (2026)
- [Adapting Dense Vision-Language Relationships for Multi-label Classification with Partial Label](https://mlsanity.com/paper/2608.22313/) (2026)

## Abstract (arXiv)

Training the multi-label image recognition models with partial labels, in which merely some labels are known while others are unknown for each image, is a considerably challenging and practical task. To address this task, current algorithms mainly depend on pre-training classification or similarity models to generate pseudo labels for the unknown labels. However, these algorithms depend on sufficient multi-label annotations to train the models, leading to poor performance especially with low known label proportion. In this work, we propose to blend category-specific representation across different images to transfer information of known labels to complement unknown labels, which can get rid of pre-training models and thus does not depend on sufficient annotations. To this end, we design a unified semantic-aware representation blending (SARB) framework that exploits instance-level and prototype-level semantic representation to complement unknown labels by two complementary modules: 1) an instance-level representation blending (ILRB) module blends the representations of the known labels in an image to the representations of the unknown labels in another image to complement these unknown labels. 2) a prototype-level representation blending (PLRB) module learns more stable representation prototypes for each category and blends the representation of unknown labels with the prototypes of corresponding labels to complement these labels. Extensive experiments on the MS-COCO, Visual Genome, Pascal VOC 2007 datasets show that the proposed SARB framework obtains superior performance over current leading competitors on all known label proportion settings, i.e., with the mAP improvement of 4.6%, 4.%, 2.2% on these three datasets when the known label proportion is 10%. Codes are available at https://github.com/HCPLab-SYSU/HCP-MLR-PL.