# Multi-Label Classification with Label Graph Superimposing

arXiv: https://arxiv.org/abs/1911.09243 · HTML: https://mlsanity.com/paper/1911.09243/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2019.

## Built on by (most cited first, 30 of 65)

- [DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2206.09541/) (2022)
- [CholecTriplet2021: A benchmark challenge for surgical action triplet recognition](https://mlsanity.com/paper/2204.04746/) (2022)
- [Transformer-based Dual Relation Graph for Multi-label Image Recognition](https://mlsanity.com/paper/2110.04722/) (2021)
- [Semantic Diversity Learning for Zero-Shot Multi-label Classification](https://mlsanity.com/paper/2105.05926/) (2021)
- [Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels](https://mlsanity.com/paper/2303.13223/) (2023)
- [DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2308.01890/) (2023)
- [Deep Learning for Multi-Label Learning: A Comprehensive Survey](https://mlsanity.com/paper/2401.16549/) (2024)
- [Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark](https://mlsanity.com/paper/2103.10895/) (2021)
- [Spatial Consistency Loss for Training Multi-Label Classifiers from Single-Label Annotations](https://mlsanity.com/paper/2203.06127/) (2022)
- [Inferring Prototypes for Multi-Label Few-Shot Image Classification with Word Vector Guided Attention](https://mlsanity.com/paper/2112.01037/) (2021)
- [DICNet: Deep Instance-Level Contrastive Network for Double Incomplete Multi-View Multi-Label Classification](https://mlsanity.com/paper/2303.08358/) (2023)
- [SemiGNN-PPI: Self-Ensembling Multi-Graph Neural Network for Efficient and Generalizable Protein-Protein Interaction Prediction](https://mlsanity.com/paper/2305.08316/) (2023)
- [GM-MLIC: Graph Matching based Multi-Label Image Classification](https://mlsanity.com/paper/2104.14762/) (2021)
- [Graph Attention Transformer Network for Multi-Label Image Classification](https://mlsanity.com/paper/2203.04049/) (2022)
- [Correlation-Aware Graph Convolutional Networks for Multi-Label Node Classification](https://mlsanity.com/paper/2411.17350/) (2024)
- [Evaluating Multi-label Classifiers with Noisy Labels](https://mlsanity.com/paper/2102.08427/) (2021)
- [Polynomial Composition Activations: Unleashing the Dynamics of Large Language Models](https://mlsanity.com/paper/2411.03884/) (2024)
- [Learning from Label Relationships in Human Affect](https://mlsanity.com/paper/2207.05577/) (2022)
- [End-to-End Supervised Multilabel Contrastive Learning](https://mlsanity.com/paper/2307.03967/) (2023)
- [Data-free Multi-label Image Recognition via LLM-powered Prompt Tuning](https://mlsanity.com/paper/2403.01209/) (2024)
- [Counterfactual Reasoning for Multi-Label Image Classification via Patching-Based Training](https://mlsanity.com/paper/2404.06287/) (2024)
- [Unifying Relational Sentence Generation and Retrieval for Medical Image Report Composition](https://mlsanity.com/paper/2101.03287/) (2021)
- [Rethinking Crowdsourcing Annotation: Partial Annotation with Salient Labels for Multi-Label Image Classification](https://mlsanity.com/paper/2109.02688/) (2021)
- [Semantic-Aware Graph Matching Mechanism for Multi-Label Image Recognition](https://mlsanity.com/paper/2304.11275/) (2023)
- [Multi-layered Semantic Representation Network for Multi-label Image Classification](https://mlsanity.com/paper/2106.11596/) (2021)
- [GraSSNet: Graph Soft Sensing Neural Networks](https://mlsanity.com/paper/2111.06980/) (2021)
- [Towards Trustworthy Multi-label Sewer Defect Classification via Evidential Deep Learning](https://mlsanity.com/paper/2210.13782/) (2022)
- [Globally Correlation-Aware Hard Negative Generation](https://mlsanity.com/paper/2411.13145/) (2024)
- [Spatial-context-aware deep neural network for multi-class image classification](https://mlsanity.com/paper/2111.12296/) (2021)
- [Estimator: An Effective and Scalable Framework for Transportation Mode Classification over Trajectories](https://mlsanity.com/paper/2212.05502/) (2022)

## Abstract (arXiv)

Images or videos always contain multiple objects or actions. Multi-label recognition has been witnessed to achieve pretty performance attribute to the rapid development of deep learning technologies. Recently, graph convolution network (GCN) is leveraged to boost the performance of multi-label recognition. However, what is the best way for label correlation modeling and how feature learning can be improved with label system awareness are still unclear. In this paper, we propose a label graph superimposing framework to improve the conventional GCN+CNN framework developed for multi-label recognition in the following two aspects. Firstly, we model the label correlations by superimposing label graph built from statistical co-occurrence information into the graph constructed from knowledge priors of labels, and then multi-layer graph convolutions are applied on the final superimposed graph for label embedding abstraction. Secondly, we propose to leverage embedding of the whole label system for better representation learning. In detail, lateral connections between GCN and CNN are added at shallow, middle and deep layers to inject information of label system into backbone CNN for label-awareness in the feature learning process. Extensive experiments are carried out on MS-COCO and Charades datasets, showing that our proposed solution can greatly improve the recognition performance and achieves new state-of-the-art recognition performance.