# Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels

arXiv: https://arxiv.org/abs/2303.13223 · HTML: https://mlsanity.com/paper/2303.13223/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 23 Mar 2023.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [How Can We Know What Language Models Know?](https://mlsanity.com/paper/1911.12543/) (2019)
- [FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling](https://mlsanity.com/paper/2110.08263/) (2021)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://mlsanity.com/paper/2010.15980/) (2020)
- [Learning a Deep ConvNet for Multi-label Classification with Partial Labels](https://mlsanity.com/paper/1902.09720/) (2019)
- [Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition](https://mlsanity.com/paper/1908.07325/) (2019)
- [DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2206.09541/) (2022)
- [Multi-Label Learning from Single Positive Labels](https://mlsanity.com/paper/2106.09708/) (2021)
- [Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition](https://mlsanity.com/paper/2009.09450/) (2020)
- [Multi-Label Classification with Label Graph Superimposing](https://mlsanity.com/paper/1911.09243/) (2019)
- [Orderless Recurrent Models for Multi-label Classification](https://mlsanity.com/paper/1911.09996/) (2019)
- [Structured Semantic Transfer for Multi-Label Recognition with Partial Labels](https://mlsanity.com/paper/2112.10941/) (2021)
- [Large Loss Matters in Weakly Supervised Multi-Label Classification](https://mlsanity.com/paper/2206.03740/) (2022)
- [Simple and Robust Loss Design for Multi-Label Learning with Missing Labels](https://mlsanity.com/paper/2112.07368/) (2021)
- [Multi-label Classification with Partial Annotations using Class-aware Selective Loss](https://mlsanity.com/paper/2110.10955/) (2021)
- [DeepStyle: Multimodal Search Engine for Fashion and Interior Design](https://mlsanity.com/paper/1801.03002/) (2018)

## Built on by (most cited first, 24 of 24)

- [RepViT: Revisiting Mobile CNN From ViT Perspective](https://mlsanity.com/paper/2307.09283/) (2023)
- [Prompt Engineering for Healthcare: Methodologies and Applications](https://mlsanity.com/paper/2304.14670/) (2023)
- [DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2308.01890/) (2023)
- [PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation](https://mlsanity.com/paper/2403.09192/) (2024)
- [LSNet: See Large, Focus Small](https://mlsanity.com/paper/2503.23135/) (2025)
- [PrefixKV: Adaptive Prefix KV Cache is What Vision Instruction-Following Models Need for Efficient Generation](https://mlsanity.com/paper/2412.03409/) (2024)
- [Vision-Language Pseudo-Labels for Single-Positive Multi-Label Learning](https://mlsanity.com/paper/2310.15985/) (2023)
- [Text-Region Matching for Multi-Label Image Recognition with Missing Labels](https://mlsanity.com/paper/2407.18520/) (2024)
- [Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations](https://mlsanity.com/paper/2507.09500/) (2025)
- [Improving Multi-label Recognition using Class Co-Occurrence Probabilities](https://mlsanity.com/paper/2404.16193/) (2024)
- [Category-wise Fine-Tuning: Resisting Incorrect Pseudo-Labels in Multi-Label Image Classification with Partial Labels](https://mlsanity.com/paper/2401.16991/) (2024)
- [Rethinking Prompting Strategies for Multi-Label Recognition with Partial Annotations](https://mlsanity.com/paper/2409.08381/) (2024)
- [Efficiently Disentangling CLIP for Multi-Object Perception](https://mlsanity.com/paper/2502.02977/) (2025)
- [Exploring Partial Multi-Label Learning via Integrating Semantic Co-occurrence Knowledge](https://mlsanity.com/paper/2507.05992/) (2025)
- [More Reliable Pseudo-labels, Better Performance: A Generalized Approach to Single Positive Multi-label Learning](https://mlsanity.com/paper/2508.20381/) (2025)
- [Unleashing the Power of Vision-Language Models for Long-Tailed Multi-Label Visual Recognition](https://mlsanity.com/paper/2511.20641/) (2025)
- [Determined Multi-Label Learning via Similarity-Based Prompt](https://mlsanity.com/paper/2403.16482/) (2024)
- [Free Performance Gain from Mixing Multiple Partially Labeled Samples in Multi-label Image Classification](https://mlsanity.com/paper/2405.15860/) (2024)
- [A Survey on Incomplete Multi-label Learning: Recent Advances and Future Trends](https://mlsanity.com/paper/2406.06119/) (2024)
- [The Demon is in Ambiguity: Revisiting Situation Recognition with Single Positive Multi-Label Learning](https://mlsanity.com/paper/2508.21816/) (2025)
- [Incomplete Multi-Label Image Recognition by Co-learning Semantic-Aware Features and Label Recovery](https://mlsanity.com/paper/2510.10055/) (2025)
- [Hyperbolic Structured Classification for Robust Single Positive Multi-label Learning](https://mlsanity.com/paper/2510.15296/) (2025)
- [FedMPT: Federated Multi-label Prompt Tuning of Vision-Language Models](https://mlsanity.com/paper/2605.28347/) (2026)
- [Towards Purified Multi-Label Test-Time Adaptation of Vision-Language Models](https://mlsanity.com/paper/2608.25653/) (2026)

## Abstract (arXiv)

Multi-label recognition (MLR) with incomplete labels is very challenging. Recent works strive to explore the image-to-label correspondence in the vision-language model, ie, CLIP, to compensate for insufficient annotations. In spite of promising performance, they generally overlook the valuable prior about the label-to-label correspondence. In this paper, we advocate remedying the deficiency of label supervision for the MLR with incomplete labels by deriving a structured semantic prior about the label-to-label correspondence via a semantic prior prompter. We then present a novel Semantic Correspondence Prompt Network (SCPNet), which can thoroughly explore the structured semantic prior. A Prior-Enhanced Self-Supervised Learning method is further introduced to enhance the use of the prior. Comprehensive experiments and analyses on several widely used benchmark datasets show that our method significantly outperforms existing methods on all datasets, well demonstrating the effectiveness and the superiority of our method. Our code will be available at https://github.com/jameslahm/SCPNet.