# Improving Multi-label Recognition using Class Co-Occurrence Probabilities

arXiv: https://arxiv.org/abs/2404.16193 · HTML: https://mlsanity.com/paper/2404.16193/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 54 references that are in the map.
- First posted on arXiv 24 Apr 2024.

## Builds on (most cited first, 30 of 39)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification](https://mlsanity.com/paper/2207.09519/) (2022)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model](https://mlsanity.com/paper/2112.14757/) (2021)
- [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016)
- [CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models](https://mlsanity.com/paper/2109.11797/) (2021)
- [Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition](https://mlsanity.com/paper/1908.07325/) (2019)
- [Vision-by-Language for Training-Free Compositional Image Retrieval](https://mlsanity.com/paper/2310.09291/) (2023)
- [DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2206.09541/) (2022)
- [The Emerging Trends of Multi-Label Learning](https://mlsanity.com/paper/2011.11197/) (2020)
- [Influence-Balanced Loss for Imbalanced Visual Classification](https://mlsanity.com/paper/2110.02444/) (2021)
- [Multi-Label Learning from Single Positive Labels](https://mlsanity.com/paper/2106.09708/) (2021)
- [Striking the Right Balance with Uncertainty](https://mlsanity.com/paper/1901.07590/) (2019)
- [Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition](https://mlsanity.com/paper/2009.09450/) (2020)
- [Learning Fast Sample Re-weighting Without Reward Data](https://mlsanity.com/paper/2109.03216/) (2021)
- [Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels](https://mlsanity.com/paper/1512.06974/) (2015)
- [Orderless Recurrent Models for Multi-label Classification](https://mlsanity.com/paper/1911.09996/) (2019)

## Built on by (most cited first, 6 of 6)

- [Potential Field Based Deep Metric Learning](https://mlsanity.com/paper/2405.18560/) (2024)
- [SPARC: Score Prompting and Adaptive Fusion for Zero-Shot Multi-Label Recognition in Vision-Language Models](https://mlsanity.com/paper/2502.16911/) (2025)
- [Rethinking Prompting Strategies for Multi-Label Recognition with Partial Annotations](https://mlsanity.com/paper/2409.08381/) (2024)
- [Efficiently Disentangling CLIP for Multi-Object Perception](https://mlsanity.com/paper/2502.02977/) (2025)
- [Any-Class Presence Likelihood for Robust Multi-Label Classification with Abundant Negative Data](https://mlsanity.com/paper/2506.05721/) (2025)
- [Rethinking Multi-Label Image Classification With Deep Learning: Taxonomy, Challenge, and Outlook](https://mlsanity.com/paper/2607.00839/) (2026)

## Abstract (arXiv)

Multi-label Recognition (MLR) involves the identification of multiple objects within an image. To address the additional complexity of this problem, recent works have leveraged information from vision-language models (VLMs) trained on large text-images datasets for the task. These methods learn an independent classifier for each object (class), overlooking correlations in their occurrences. Such co-occurrences can be captured from the training data as conditional probabilities between a pair of classes. We propose a framework to extend the independent classifiers by incorporating the co-occurrence information for object pairs to improve the performance of independent classifiers. We use a Graph Convolutional Network (GCN) to enforce the conditional probabilities between classes, by refining the initial estimates derived from image and text sources obtained using VLMs. We validate our method on four MLR datasets, where our approach outperforms all state-of-the-art methods.