# Transformer-based Multi-Instance Learning for Weakly Supervised Object Detection

arXiv: https://arxiv.org/abs/2303.14999 · HTML: https://mlsanity.com/paper/2303.14999/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 27 Mar 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [On the Relationship between Self-Attention and Convolutional Layers](https://mlsanity.com/paper/1911.03584/) (2019)
- [Weakly Supervised Deep Detection Networks](https://mlsanity.com/paper/1511.02853/) (2015)
- [An Improved Analysis of Stochastic Gradient Descent with Momentum](https://mlsanity.com/paper/2007.07989/) (2020)
- [Revisiting Multiple Instance Neural Networks](https://mlsanity.com/paper/1610.02501/) (2016)
- [Multiple Instance Detection Network with Online Instance Classifier Refinement](https://mlsanity.com/paper/1704.00138/) (2017)
- [PCL: Proposal Cluster Learning for Weakly Supervised Object Detection](https://mlsanity.com/paper/1807.03342/) (2018)
- [Instance-aware, Context-focused, and Memory-efficient Weakly Supervised Object Detection](https://mlsanity.com/paper/2004.04725/) (2020)
- [C-MIL: Continuation Multiple Instance Learning for Weakly Supervised Object Detection](https://mlsanity.com/paper/1904.05647/) (2019)
- [WSOD^2: Learning Bottom-up and Top-down Objectness Distillation for Weakly-supervised Object Detection](https://mlsanity.com/paper/1909.04972/) (2019)
- [Comprehensive Attention Self-Distillation for Weakly-Supervised Object Detection](https://mlsanity.com/paper/2010.12023/) (2020)
- [Min-Entropy Latent Model for Weakly Supervised Object Detection](https://mlsanity.com/paper/1902.06057/) (2019)
- [SLV: Spatial Likelihood Voting for Weakly Supervised Object Detection](https://mlsanity.com/paper/2006.12884/) (2020)
- [Object Instance Mining for Weakly Supervised Object Detection](https://mlsanity.com/paper/2002.01087/) (2020)
- [Multi-Instance Causal Representation Learning for Instance Label Prediction and Out-of-Distribution Generalization](https://mlsanity.com/paper/2202.12570/) (2022)
- [Object Discovery via Contrastive Learning for Weakly Supervised Object Detection](https://mlsanity.com/paper/2208.07576/) (2022)
- [W2N:Switching From Weak Supervision to Noisy Supervision for Object Detection](https://mlsanity.com/paper/2207.12104/) (2022)
- [Salvage of Supervision in Weakly Supervised Object Detection](https://mlsanity.com/paper/2106.04073/) (2021)
- [Non-I.I.D. Multi-Instance Learning for Predicting Instance and Bag Labels using Variational Auto-Encoder](https://mlsanity.com/paper/2105.01276/) (2021)

## Abstract (arXiv)

Weakly Supervised Object Detection (WSOD) enables the training of object detection models using only image-level annotations. State-of-the-art WSOD detectors commonly rely on multi-instance learning (MIL) as the backbone of their detectors and assume that the bounding box proposals of an image are independent of each other. However, since such approaches only utilize the highest score proposal and discard the potentially useful information from other proposals, their independent MIL backbone often limits models to salient parts of an object or causes them to detect only one object per class. To solve the above problems, we propose a novel backbone for WSOD based on our tailored Vision Transformer named Weakly Supervised Transformer Detection Network (WSTDN). Our algorithm is not only the first to demonstrate that self-attention modules that consider inter-instance relationships are effective backbones for WSOD, but also we introduce a novel bounding box mining method (BBM) integrated with a memory transfer refinement (MTR) procedure to utilize the instance dependencies for facilitating instance refinements. Experimental results on PASCAL VOC2007 and VOC2012 benchmarks demonstrate the effectiveness of our proposed WSTDN and modified instance refinement modules.