# Transformer-based Dual Relation Graph for Multi-label Image Recognition

arXiv: https://arxiv.org/abs/2110.04722 · HTML: https://mlsanity.com/paper/2110.04722/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 48 references that are in the map.
- First posted on arXiv 10 Oct 2021.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [A Decomposable Attention Model for Natural Language Inference](https://mlsanity.com/paper/1606.01933/) (2016)
- [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016)
- [General Multi-label Image Classification with Transformers](https://mlsanity.com/paper/2011.14027/) (2020)
- [Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition](https://mlsanity.com/paper/1908.07325/) (2019)
- [Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification](https://mlsanity.com/paper/1702.05891/) (2017)
- [Multi-label Image Recognition by Recurrently Discovering Attentional Regions](https://mlsanity.com/paper/1711.02816/) (2017)
- [Attention-Driven Dynamic Graph Convolutional Network for Multi-Label Image Recognition](https://mlsanity.com/paper/2012.02994/) (2020)
- [Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification](https://mlsanity.com/paper/1912.07872/) (2019)
- [Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recognition](https://mlsanity.com/paper/2009.09450/) (2020)
- [Multi-Label Classification with Label Graph Superimposing](https://mlsanity.com/paper/1911.09243/) (2019)
- [CNN: Single-label to Multi-label](https://mlsanity.com/paper/1406.5726/) (2014)
- [Learning to Discover Multi-Class Attentional Regions for Multi-Label Image Recognition](https://mlsanity.com/paper/2007.01755/) (2020)
- [Orderless Recurrent Models for Multi-label Classification](https://mlsanity.com/paper/1911.09996/) (2019)
- [Recurrent Attentional Reinforcement Learning for Multi-label Image Recognition](https://mlsanity.com/paper/1712.07465/) (2017)
- [Exploit Bounding Box Annotations for Multi-label Object Recognition](https://mlsanity.com/paper/1504.05843/) (2015)
- [Multi-Evidence Filtering and Fusion for Multi-Label Classification, Object Detection and Semantic Segmentation Based on Weakly Supervised Learning](https://mlsanity.com/paper/1802.09129/) (2018)
- [Order-Free RNN with Visual Attention for Multi-Label Classification](https://mlsanity.com/paper/1707.05495/) (2017)

## Built on by (most cited first, 30 of 41)

- [BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation](https://mlsanity.com/paper/2204.00987/) (2022)
- [The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation](https://mlsanity.com/paper/2306.01923/) (2023)
- [Monocular Depth Estimation using Diffusion Models](https://mlsanity.com/paper/2302.14816/) (2023)
- [A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective](https://mlsanity.com/paper/2209.13232/) (2022)
- [SQLdepth: Generalizable Self-Supervised Fine-Structured Monocular Depth Estimation](https://mlsanity.com/paper/2309.00526/) (2023)
- [Multi-label Cluster Discrimination for Visual Representation Learning](https://mlsanity.com/paper/2407.17331/) (2024)
- [Test-Time Backdoor Detection for Object Detection Models](https://mlsanity.com/paper/2503.15293/) (2025)
- [End-to-End Supervised Multilabel Contrastive Learning](https://mlsanity.com/paper/2307.03967/) (2023)
- [Semantic-Aware Dual Contrastive Learning for Multi-label Image Classification](https://mlsanity.com/paper/2307.09715/) (2023)
- [SPIdepth: Strengthened Pose Information for Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2404.12501/) (2024)
- [LP-DETR: Layer-wise Progressive Relations for Object Detection](https://mlsanity.com/paper/2502.05147/) (2025)
- [Knowledge Restore and Transfer for Multi-label Class-Incremental Learning](https://mlsanity.com/paper/2302.13334/) (2023)
- [Positive Label Is All You Need for Multi-Label Classification](https://mlsanity.com/paper/2306.16016/) (2023)
- [Semantic Representation and Dependency Learning for Multi-Label Image Recognition](https://mlsanity.com/paper/2204.03795/) (2022)
- [Confidence Self-Calibration for Multi-Label Class-Incremental Learning](https://mlsanity.com/paper/2403.12559/) (2024)
- [Semantic Contrastive Bootstrapping for Single-positive Multi-label Recognition](https://mlsanity.com/paper/2307.07680/) (2023)
- [SpliceMix: A Cross-scale and Semantic Blending Augmentation Strategy for Multi-label Image Classification](https://mlsanity.com/paper/2311.15200/) (2023)
- [MambaDepth: Enhancing Long-range Dependency for Self-Supervised Fine-Structured Monocular Depth Estimation](https://mlsanity.com/paper/2406.04532/) (2024)
- [Curiosity Meets Cooperation: A Game-Theoretic Approach to Long-Tail Multi-Label Learning](https://mlsanity.com/paper/2510.17520/) (2025)
- [GKGNet: Group K-Nearest Neighbor based Graph Convolutional Network for Multi-Label Image Recognition](https://mlsanity.com/paper/2308.14378/) (2023)
- [ProbMCL: Simple Probabilistic Contrastive Learning for Multi-label Visual Classification](https://mlsanity.com/paper/2401.01448/) (2024)
- [TIE-KD: Teacher-Independent and Explainable Knowledge Distillation for Monocular Depth Estimation](https://mlsanity.com/paper/2402.14340/) (2024)
- [SSPA: Split-and-Synthesize Prompting with Gated Alignments for Multi-Label Image Recognition](https://mlsanity.com/paper/2407.20920/) (2024)
- [Learning Disentangled Label Representations for Multi-label Classification](https://mlsanity.com/paper/2212.01461/) (2022)
- [Depth-Relative Self Attention for Monocular Depth Estimation](https://mlsanity.com/paper/2304.12849/) (2023)
- [Learning to recognize occluded and small objects with partial inputs](https://mlsanity.com/paper/2310.18517/) (2023)
- [Multi-Grained Compositional Visual Clue Learning for Image Intent Recognition](https://mlsanity.com/paper/2504.18201/) (2025)
- [Dynamic Graph Induced Contour-aware Heat Conduction Network for Event-based Object Detection](https://mlsanity.com/paper/2505.12908/) (2025)
- [Text to Image for Multi-Label Image Recognition with Joint Prompt-Adapter Learning](https://mlsanity.com/paper/2506.10575/) (2025)
- [Efficient Image-Text Retrieval via Keyword-Guided Pre-Screening](https://mlsanity.com/paper/2303.07740/) (2023)

## Abstract (arXiv)

The simultaneous recognition of multiple objects in one image remains a challenging task, spanning multiple events in the recognition field such as various object scales, inconsistent appearances, and confused inter-class relationships. Recent research efforts mainly resort to the statistic label co-occurrences and linguistic word embedding to enhance the unclear semantics. Different from these researches, in this paper, we propose a novel Transformer-based Dual Relation learning framework, constructing complementary relationships by exploring two aspects of correlation, i.e., structural relation graph and semantic relation graph. The structural relation graph aims to capture long-range correlations from object context, by developing a cross-scale transformer-based architecture. The semantic graph dynamically models the semantic meanings of image objects with explicit semantic-aware constraints. In addition, we also incorporate the learnt structural relationship into the semantic graph, constructing a joint relation graph for robust representations. With the collaborative learning of these two effective relation graphs, our approach achieves new state-of-the-art on two popular multi-label recognition benchmarks, i.e., MS-COCO and VOC 2007 dataset.