# Semantic-guided Representation Learning for Multi-Label Recognition

arXiv: https://arxiv.org/abs/2504.03801 · HTML: https://mlsanity.com/paper/2504.03801/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 24 references that are in the map.
- First posted on arXiv 4 Apr 2025.

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model](https://mlsanity.com/paper/2203.14940/) (2022)
- [Bilinear Attention Networks](https://mlsanity.com/paper/1805.07932/) (2018)
- [Label-Embedding for Image Classification](https://mlsanity.com/paper/1503.08677/) (2015)
- [DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2206.09541/) (2022)
- [Multi-Label Learning from Single Positive Labels](https://mlsanity.com/paper/2106.09708/) (2021)
- [Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognition](https://mlsanity.com/paper/2012.02994/) (2020)
- [Open-Vocabulary Multi-Label Classification via Multi-Modal Knowledge Transfer](https://mlsanity.com/paper/2207.01887/) (2022)
- [Discriminative Region-based Multi-Label Zero-Shot Learning](https://mlsanity.com/paper/2108.09301/) (2021)
- [Semantic Diversity Learning for Zero-Shot Multi-label Classification](https://mlsanity.com/paper/2105.05926/) (2021)
- [Acknowledging the Unknown for Multi-label Learning with Single Positive Labels](https://mlsanity.com/paper/2203.16219/) (2022)
- [Fast Zero-Shot Image Tagging](https://mlsanity.com/paper/1605.09759/) (2016)
- [Generative Multi-Label Zero-Shot Learning](https://mlsanity.com/paper/2101.11606/) (2021)
- [Class-Distribution-Aware Pseudo Labeling for Semi-Supervised Multi-Label Learning](https://mlsanity.com/paper/2305.02795/) (2023)
- [Vision-Language Pseudo-Labels for Single-Positive Multi-Label Learning](https://mlsanity.com/paper/2310.15985/) (2023)

## Built on by (most cited first, 2 of 2)

- [The Demon is in Ambiguity: Revisiting Situation Recognition with Single Positive Multi-Label Learning](https://mlsanity.com/paper/2508.21816/) (2025)
- [Hyperbolic Structured Classification for Robust Single Positive Multi-label Learning](https://mlsanity.com/paper/2510.15296/) (2025)

## Abstract (arXiv)

Multi-label Recognition (MLR) involves assigning multiple labels to each data instance in an image, offering advantages over single-label classification in complex scenarios. However, it faces the challenge of annotating all relevant categories, often leading to uncertain annotations, such as unseen or incomplete labels. Recent Vision and Language Pre-training (VLP) based methods have made significant progress in tackling zero-shot MLR tasks by leveraging rich vision-language correlations. However, the correlation between multi-label semantics has not been fully explored, and the learned visual features often lack essential semantic information. To overcome these limitations, we introduce a Semantic-guided Representation Learning approach (SigRL) that enables the model to learn effective visual and textual representations, thereby improving the downstream alignment of visual images and categories. Specifically, we first introduce a graph-based multi-label correlation module (GMC) to facilitate information exchange between labels, enriching the semantic representation across the multi-label texts. Next, we propose a Semantic Visual Feature Reconstruction module (SVFR) to enhance the semantic information in the visual representation by integrating the learned textual representation during reconstruction. Finally, we optimize the image-text matching capability of the VLP model using both local and global features to achieve zero-shot MLR. Comprehensive experiments are conducted on several MLR benchmarks, encompassing both zero-shot MLR (with unseen labels) and single positive multi-label learning (with limited labels), demonstrating the superior performance of our approach compared to state-of-the-art methods. The code is available at https://github.com/MVL-Lab/SigRL.