# CNN-RNN: A Unified Framework for Multi-label Image Classification

arXiv: https://arxiv.org/abs/1604.04573 · HTML: https://mlsanity.com/paper/1604.04573/

## Key facts

- Cited by 190 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2016.

## Built on by (most cited first, 30 of 190)

- [Query2Label: A Simple Transformer Way to Multi-Label Classification](https://mlsanity.com/paper/2107.10834/) (2021)
- [DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2206.09541/) (2022)
- [Can multi-label classification networks know what they don't know?](https://mlsanity.com/paper/2109.14162/) (2021)
- [Multi-Label Learning from Single Positive Labels](https://mlsanity.com/paper/2106.09708/) (2021)
- [Residual Attention: A Simple but Effective Method for Multi-Label Recognition](https://mlsanity.com/paper/2108.02456/) (2021)
- [Texts as Images in Prompt Tuning for Multi-Label Image Recognition](https://mlsanity.com/paper/2211.12739/) (2022)
- [Transformer-based Dual Relation Graph for Multi-label Image Recognition](https://mlsanity.com/paper/2110.04722/) (2021)
- [Structured Semantic Transfer for Multi-Label Recognition with Partial Labels](https://mlsanity.com/paper/2112.10941/) (2021)
- [Large Loss Matters in Weakly Supervised Multi-Label Classification](https://mlsanity.com/paper/2206.03740/) (2022)
- [MAT-Agent: Adaptive Multi-Agent Training Optimization](https://mlsanity.com/paper/2510.17845/) (2025)
- [Open-Vocabulary Multi-Label Classification via Multi-Modal Knowledge Transfer](https://mlsanity.com/paper/2207.01887/) (2022)
- [Modeling Multi-Label Action Dependencies for Temporal Action Localization](https://mlsanity.com/paper/2103.03027/) (2021)
- [Discriminative Region-based Multi-Label Zero-Shot Learning](https://mlsanity.com/paper/2108.09301/) (2021)
- [Multi-label Classification with Partial Annotations using Class-aware Selective Loss](https://mlsanity.com/paper/2110.10955/) (2021)
- [Acknowledging the Unknown for Multi-label Learning with Single Positive Labels](https://mlsanity.com/paper/2203.16219/) (2022)
- [sigmoidF1: A Smooth F1 Score Surrogate Loss for Multilabel Classification](https://mlsanity.com/paper/2108.10566/) (2021)
- [DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2308.01890/) (2023)
- [Meta-Learning for Multi-Label Few-Shot Classification](https://mlsanity.com/paper/2110.13494/) (2021)
- [Towards Unbiased Multi-label Zero-Shot Learning with Pyramid and Semantic Attention](https://mlsanity.com/paper/2203.03483/) (2022)
- [PatchCT: Aligning Patch Set and Label Set with Conditional Transport for Multi-Label Image Classification](https://mlsanity.com/paper/2307.09066/) (2023)
- [Generative Multi-Label Zero-Shot Learning](https://mlsanity.com/paper/2101.11606/) (2021)
- [Heterogeneous Semantic Transfer for Multi-label Recognition with Partial Labels](https://mlsanity.com/paper/2205.11131/) (2022)
- [Retinal Vessel Segmentation via a Multi-resolution Contextual Network and Adversarial Learning](https://mlsanity.com/paper/2304.12856/) (2023)
- [Banana: Banach Fixed-Point Network for Pointcloud Segmentation with Inter-Part Equivariance](https://mlsanity.com/paper/2305.16314/) (2023)
- [Deep Learning for Multi-Label Learning: A Comprehensive Survey](https://mlsanity.com/paper/2401.16549/) (2024)
- [Multi-Label Image Classification with Contrastive Learning](https://mlsanity.com/paper/2107.11626/) (2021)
- [Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark](https://mlsanity.com/paper/2103.10895/) (2021)
- [AGCN: Augmented Graph Convolutional Network for Lifelong Multi-label Image Recognition](https://mlsanity.com/paper/2203.05534/) (2022)
- [ZLPR: A Novel Loss for Multi-label Classification](https://mlsanity.com/paper/2208.02955/) (2022)
- [Gaussian Mixture Variational Autoencoder with Contrastive Learning for Multi-Label Classification](https://mlsanity.com/paper/2112.00976/) (2021)

## Abstract (arXiv)

While deep convolutional neural networks (CNNs) have shown a great success in single-label image classification, it is important to note that real world images generally contain multiple labels, which could correspond to different objects, scenes, actions and attributes in an image. Traditional approaches to multi-label image classification learn independent classifiers for each category and employ ranking or thresholding on the classification results. These techniques, although working well, fail to explicitly exploit the label dependencies in an image. In this paper, we utilize recurrent neural networks (RNNs) to address this problem. Combined with CNNs, the proposed CNN-RNN framework learns a joint image-label embedding to characterize the semantic label dependency as well as the image-label relevance, and it can be trained end-to-end from scratch to integrate both information in a unified framework. Experimental results on public benchmark datasets demonstrate that the proposed architecture achieves better performance than the state-of-the-art multi-label classification model