# Exploit Bounding Box Annotations for Multi-label Object Recognition

arXiv: https://arxiv.org/abs/1504.05843 · HTML: https://mlsanity.com/paper/1504.05843/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Apr 2015.

## Built on by (most cited first, 30 of 39)

- [Query2Label: A Simple Transformer Way to Multi-Label Classification](https://mlsanity.com/paper/2107.10834/) (2021)
- [ML-Decoder: Scalable and Versatile Classification Head](https://mlsanity.com/paper/2111.12933/) (2021)
- [Residual Attention: A Simple but Effective Method for Multi-Label Recognition](https://mlsanity.com/paper/2108.02456/) (2021)
- [Texts as Images in Prompt Tuning for Multi-Label Image Recognition](https://mlsanity.com/paper/2211.12739/) (2022)
- [Transformer-based Dual Relation Graph for Multi-label Image Recognition](https://mlsanity.com/paper/2110.04722/) (2021)
- [Structured Semantic Transfer for Multi-Label Recognition with Partial Labels](https://mlsanity.com/paper/2112.10941/) (2021)
- [Acknowledging the Unknown for Multi-label Learning with Single Positive Labels](https://mlsanity.com/paper/2203.16219/) (2022)
- [Meta-Learning for Multi-Label Few-Shot Classification](https://mlsanity.com/paper/2110.13494/) (2021)
- [Heterogeneous Semantic Transfer for Multi-label Recognition with Partial Labels](https://mlsanity.com/paper/2205.11131/) (2022)
- [Deep Learning for Multi-Label Learning: A Comprehensive Survey](https://mlsanity.com/paper/2401.16549/) (2024)
- [Sewer-ML: A Multi-Label Sewer Defect Classification Dataset and Benchmark](https://mlsanity.com/paper/2103.10895/) (2021)
- [Multi-label Cluster Discrimination for Visual Representation Learning](https://mlsanity.com/paper/2407.17331/) (2024)
- [Dynamic Correlation Learning and Regularization for Multi-Label Confidence Calibration](https://mlsanity.com/paper/2407.06844/) (2024)
- [GM-MLIC: Graph Matching based Multi-Label Image Classification](https://mlsanity.com/paper/2104.14762/) (2021)
- [Multi-Label Continual Learning using Augmented Graph Convolutional Network](https://mlsanity.com/paper/2211.14763/) (2022)
- [Graph Attention Transformer Network for Multi-Label Image Classification](https://mlsanity.com/paper/2203.04049/) (2022)
- [MultiGuard: Provably Robust Multi-label Classification against Adversarial Examples](https://mlsanity.com/paper/2210.01111/) (2022)
- [CLIP-Decoder : ZeroShot Multilabel Classification using Multimodal CLIP Aligned Representation](https://mlsanity.com/paper/2406.14830/) (2024)
- [End-to-End Supervised Multilabel Contrastive Learning](https://mlsanity.com/paper/2307.03967/) (2023)
- [Semantic-Aware Dual Contrastive Learning for Multi-label Image Classification](https://mlsanity.com/paper/2307.09715/) (2023)
- [Counterfactual Reasoning for Multi-Label Image Classification via Patching-Based Training](https://mlsanity.com/paper/2404.06287/) (2024)
- [Semantic-Aware Graph Matching Mechanism for Multi-Label Image Recognition](https://mlsanity.com/paper/2304.11275/) (2023)
- [Multi-layered Semantic Representation Network for Multi-label Image Classification](https://mlsanity.com/paper/2106.11596/) (2021)
- [Open Vocabulary Multi-Label Classification with Dual-Modal Decoder on Aligned Visual-Textual Features](https://mlsanity.com/paper/2208.09562/) (2022)
- [Learning Discriminative Representations for Multi-Label Image Recognition](https://mlsanity.com/paper/2107.11159/) (2021)
- [Spatial-context-aware deep neural network for multi-class image classification](https://mlsanity.com/paper/2111.12296/) (2021)
- [Multi-label Ranking: Mining Multi-label and Label Ranking Data](https://mlsanity.com/paper/2101.00583/) (2021)
- [Global Meets Local: Effective Multi-Label Image Classification via Category-Aware Weak Supervision](https://mlsanity.com/paper/2211.12716/) (2022)
- [HSVLT: Hierarchical Scale-Aware Vision-Language Transformer for Multi-Label Image Classification](https://mlsanity.com/paper/2407.16244/) (2024)
- [On the Learnability of Multilabel Ranking](https://mlsanity.com/paper/2304.03337/) (2023)

## Abstract (arXiv)

Convolutional neural networks (CNNs) have shown great performance as general feature representations for object recognition applications. However, for multi-label images that contain multiple objects from different categories, scales and locations, global CNN features are not optimal. In this paper, we incorporate local information to enhance the feature discriminative power. In particular, we first extract object proposals from each image. With each image treated as a bag and object proposals extracted from it treated as instances, we transform the multi-label recognition problem into a multi-class multi-instance learning problem. Then, in addition to extracting the typical CNN feature representation from each proposal, we propose to make use of ground-truth bounding box annotations (strong labels) to add another level of local information by using nearest-neighbor relationships of local regions to form a multi-view pipeline. The proposed multi-view multi-instance framework utilizes both weak and strong labels effectively, and more importantly it has the generalization ability to even boost the performance of unseen categories by partial strong labels from other categories. Our framework is extensively compared with state-of-the-art hand-crafted feature based methods and CNN based methods on two multi-label benchmark datasets. The experimental results validate the discriminative power and the generalization ability of the proposed framework. With strong labels, our framework is able to achieve state-of-the-art results in both datasets.