# Multi-layered Semantic Representation Network for Multi-label Image Classification

arXiv: https://arxiv.org/abs/2106.11596 · HTML: https://mlsanity.com/paper/2106.11596/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 22 Jun 2021.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Hierarchical Graph Representation Learning with Differentiable Pooling](https://mlsanity.com/paper/1806.08804/) (2018)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016)
- [Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition](https://mlsanity.com/paper/1908.07325/) (2019)
- [Unsupervised Person Re-identification via Multi-label Classification](https://mlsanity.com/paper/2004.09228/) (2020)
- [Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification](https://mlsanity.com/paper/1702.05891/) (2017)
- [A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection](https://mlsanity.com/paper/1607.07155/) (2016)
- [Multi-label Image Recognition by Recurrently Discovering Attentional Regions](https://mlsanity.com/paper/1711.02816/) (2017)
- [Learn To Pay Attention](https://mlsanity.com/paper/1804.02391/) (2018)
- [Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification](https://mlsanity.com/paper/1912.07872/) (2019)
- [Multi-Label Zero-Shot Learning with Structured Knowledge Graphs](https://mlsanity.com/paper/1711.06526/) (2017)
- [Multi-Label Classification with Label Graph Superimposing](https://mlsanity.com/paper/1911.09243/) (2019)
- [CNN: Single-label to Multi-label](https://mlsanity.com/paper/1406.5726/) (2014)
- [Recurrent Attentional Reinforcement Learning for Multi-label Image Recognition](https://mlsanity.com/paper/1712.07465/) (2017)
- [Object Detection from Video Tubelets with Convolutional Neural Networks](https://mlsanity.com/paper/1604.04053/) (2016)
- [Exploit Bounding Box Annotations for Multi-label Object Recognition](https://mlsanity.com/paper/1504.05843/) (2015)
- [Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detection](https://mlsanity.com/paper/1809.05884/) (2018)
- [Chest X-rays Classification: A Multi-Label and Fine-Grained Problem](https://mlsanity.com/paper/1807.07247/) (2018)
- [Order-Free RNN with Visual Attention for Multi-Label Classification](https://mlsanity.com/paper/1707.05495/) (2017)
- [Multi-scale recognition with DAG-CNNs](https://mlsanity.com/paper/1505.05232/) (2015)
- [One-Pass Learning with Incremental and Decremental Features](https://mlsanity.com/paper/1605.09082/) (2016)
- [Discriminative Block-Diagonal Representation Learning for Image Recognition](https://mlsanity.com/paper/1707.03548/) (2017)

## Built on by (most cited first, 6 of 6)

- [Multi-label Image Classification using Adaptive Graph Convolutional Networks: from a Single Domain to Multiple Domains](https://mlsanity.com/paper/2301.04494/) (2023)
- [All Mistakes Are Not Equal: Comprehensive Hierarchy Aware Multi-label Predictions (CHAMP)](https://mlsanity.com/paper/2206.08653/) (2022)
- [NOAH: Learning Pairwise Object Category Attentions for Image Classification](https://mlsanity.com/paper/2402.02377/) (2024)
- [Deep Dependency Networks for Multi-Label Classification](https://mlsanity.com/paper/2302.00633/) (2023)
- [Deep Dependency Networks and Advanced Inference Schemes for Multi-Label Classification](https://mlsanity.com/paper/2404.11667/) (2024)
- [Rethinking Multi-Label Image Classification With Deep Learning: Taxonomy, Challenge, and Outlook](https://mlsanity.com/paper/2607.00839/) (2026)

## Abstract (arXiv)

Multi-label image classification (MLIC) is a fundamental and practical task, which aims to assign multiple possible labels to an image. In recent years, many deep convolutional neural network (CNN) based approaches have been proposed which model label correlations to discover semantics of labels and learn semantic representations of images. This paper advances this research direction by improving both the modeling of label correlations and the learning of semantic representations. On the one hand, besides the local semantics of each label, we propose to further explore global semantics shared by multiple labels. On the other hand, existing approaches mainly learn the semantic representations at the last convolutional layer of a CNN. But it has been noted that the image representations of different layers of CNN capture different levels or scales of features and have different discriminative abilities. We thus propose to learn semantic representations at multiple convolutional layers. To this end, this paper designs a Multi-layered Semantic Representation Network (MSRN) which discovers both local and global semantics of labels through modeling label correlations and utilizes the label semantics to guide the semantic representations learning at multiple layers through an attention mechanism. Extensive experiments on four benchmark datasets including VOC 2007, COCO, NUS-WIDE, and Apparel show a competitive performance of the proposed MSRN against state-of-the-art models.