# Multi-label Image Recognition by Recurrently Discovering Attentional Regions

arXiv: https://arxiv.org/abs/1711.02816 · HTML: https://mlsanity.com/paper/1711.02816/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2017.

## Built on by (most cited first, 30 of 83)

- [Query2Label: A Simple Transformer Way to Multi-Label Classification](https://mlsanity.com/paper/2107.10834/) (2021)
- [Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels](https://mlsanity.com/paper/2101.05022/) (2021)
- [DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2206.09541/) (2022)
- [Learning to Predict Visual Attributes in the Wild](https://mlsanity.com/paper/2106.09707/) (2021)
- [ML-Decoder: Scalable and Versatile Classification Head](https://mlsanity.com/paper/2111.12933/) (2021)
- [Residual Attention: A Simple but Effective Method for Multi-Label Recognition](https://mlsanity.com/paper/2108.02456/) (2021)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Transformer-based Dual Relation Graph for Multi-label Image Recognition](https://mlsanity.com/paper/2110.04722/) (2021)
- [Structured Semantic Transfer for Multi-Label Recognition with Partial Labels](https://mlsanity.com/paper/2112.10941/) (2021)
- [Open-Vocabulary Multi-Label Classification via Multi-Modal Knowledge Transfer](https://mlsanity.com/paper/2207.01887/) (2022)
- [Modeling Multi-Label Action Dependencies for Temporal Action Localization](https://mlsanity.com/paper/2103.03027/) (2021)
- [Semantic-Aware Representation Blending for Multi-Label Image Recognition with Partial Labels](https://mlsanity.com/paper/2203.02172/) (2022)
- [Integrative Few-Shot Learning for Classification and Segmentation](https://mlsanity.com/paper/2203.15712/) (2022)
- [Semantic Diversity Learning for Zero-Shot Multi-label Classification](https://mlsanity.com/paper/2105.05926/) (2021)
- [DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2308.01890/) (2023)
- [Meta-Learning for Multi-Label Few-Shot Classification](https://mlsanity.com/paper/2110.13494/) (2021)
- [Towards Unbiased Multi-label Zero-Shot Learning with Pyramid and Semantic Attention](https://mlsanity.com/paper/2203.03483/) (2022)
- [PatchCT: Aligning Patch Set and Label Set with Conditional Transport for Multi-Label Image Classification](https://mlsanity.com/paper/2307.09066/) (2023)
- [Generative Multi-Label Zero-Shot Learning](https://mlsanity.com/paper/2101.11606/) (2021)
- [Heterogeneous Semantic Transfer for Multi-label Recognition with Partial Labels](https://mlsanity.com/paper/2205.11131/) (2022)
- [TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP Without Training](https://mlsanity.com/paper/2312.12828/) (2023)
- [Multi-Label Image Classification with Contrastive Learning](https://mlsanity.com/paper/2107.11626/) (2021)
- [Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark](https://mlsanity.com/paper/2103.10895/) (2021)
- [Inferring Prototypes for Multi-Label Few-Shot Image Classification with Word Vector Guided Attention](https://mlsanity.com/paper/2112.01037/) (2021)
- [LMPT: Prompt Tuning with Class-Specific Embedding Loss for Long-tailed Multi-Label Visual Recognition](https://mlsanity.com/paper/2305.04536/) (2023)
- [TAI++: Text as Image for Multi-Label Image Classification by Co-Learning Transferable Prompt](https://mlsanity.com/paper/2405.06926/) (2024)
- [SequencePAR: Understanding Pedestrian Attributes via A Sequence Generation Paradigm](https://mlsanity.com/paper/2312.01640/) (2023)
- [Dynamic Correlation Learning and Regularization for Multi-Label Confidence Calibration](https://mlsanity.com/paper/2407.06844/) (2024)
- [GM-MLIC: Graph Matching based Multi-Label Image Classification](https://mlsanity.com/paper/2104.14762/) (2021)
- [CheXFusion: Effective Fusion of Multi-View Features using Transformers for Long-Tailed Chest X-Ray Classification](https://mlsanity.com/paper/2308.03968/) (2023)

## Abstract (arXiv)

This paper proposes a novel deep architecture to address multi-label image recognition, a fundamental and practical task towards general visual understanding. Current solutions for this task usually rely on an extra step of extracting hypothesis regions (i.e., region proposals), resulting in redundant computation and sub-optimal performance. In this work, we achieve the interpretable and contextualized multi-label image classification by developing a recurrent memorized-attention module. This module consists of two alternately performed components: i) a spatial transformer layer to locate attentional regions from the convolutional feature maps in a region-proposal-free way and ii) an LSTM (Long-Short Term Memory) sub-network to sequentially predict semantic labeling scores on the located regions while capturing the global dependencies of these regions. The LSTM also output the parameters for computing the spatial transformer. On large-scale benchmarks of multi-label image classification (e.g., MS-COCO and PASCAL VOC 07), our approach demonstrates superior performances over other existing state-of-the-arts in both accuracy and efficiency.