# Transformer based multiple instance learning for weakly supervised histopathology image segmentation

arXiv: https://arxiv.org/abs/2205.08878 · HTML: https://mlsanity.com/paper/2205.08878/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 18 May 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Deeply-Supervised Nets](https://mlsanity.com/paper/1409.5185/) (2014)
- [Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks](https://mlsanity.com/paper/2009.11169/) (2020)
- [Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation](https://mlsanity.com/paper/1805.04574/) (2018)
- [Resnet in Resnet: Generalizing Residual Architectures](https://mlsanity.com/paper/1603.08029/) (2016)
- [DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation](https://mlsanity.com/paper/1604.02677/) (2016)
- [Multi-scale Domain-adversarial Multiple-instance CNN for Cancer Subtype Classification with Unannotated Histopathological Images](https://mlsanity.com/paper/2001.01599/) (2020)
- [Weakly Supervised Instance Segmentation using Class Peak Response](https://mlsanity.com/paper/1804.00880/) (2018)
- [Constrained Deep Weak Supervision for Histopathology Image Segmentation](https://mlsanity.com/paper/1701.00794/) (2017)
- [Multi-scale Cell Instance Segmentation with Keypoint Graph based Bounding Boxes](https://mlsanity.com/paper/1907.09140/) (2019)
- [Weakly Supervised Minirhizotron Image Segmentation with MIL-CAM](https://mlsanity.com/paper/2007.15243/) (2020)

## Built on by (most cited first, 26 of 26)

- [Interventional Bag Multi-Instance Learning On Whole-Slide Pathological Images](https://mlsanity.com/paper/2303.06873/) (2023)
- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [OCELOT: Overlapped Cell on Tissue Dataset for Histopathology](https://mlsanity.com/paper/2303.13110/) (2023)
- [A Recent Survey of Vision Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2312.00634/) (2023)
- [Prototype-Based Image Prompting for Weakly Supervised Histopathological Image Segmentation](https://mlsanity.com/paper/2503.12068/) (2025)
- [Recent Progress in Transformer-based Medical Image Analysis](https://mlsanity.com/paper/2208.06643/) (2022)
- [AME-CAM: Attentive Multiple-Exit CAM for Weakly Supervised Segmentation on MRI Brain Tumor](https://mlsanity.com/paper/2306.14505/) (2023)
- [Integrating Deep Metric Learning with Coreset for Active Learning in 3D Segmentation](https://mlsanity.com/paper/2411.15763/) (2024)
- [Cyclic Learning: Bridging Image-level Labels and Nuclei Instance Segmentation](https://mlsanity.com/paper/2306.02691/) (2023)
- [CLASS-M: Adaptive stain separation-based contrastive learning with pseudo-labeling for histopathological image classification](https://mlsanity.com/paper/2312.06978/) (2023)
- [Gall Bladder Cancer Detection from US Images with Only Image Level Labels](https://mlsanity.com/paper/2309.05261/) (2023)
- [Pan-cancer Histopathology WSI Pre-training with Position-aware Masked Autoencoder](https://mlsanity.com/paper/2407.07504/) (2024)
- [Enhancing Weakly Supervised 3D Medical Image Segmentation through Probabilistic-aware Learning](https://mlsanity.com/paper/2403.02566/) (2024)
- [Efficient Microscopic Image Instance Segmentation for Food Crystal Quality Control](https://mlsanity.com/paper/2409.18291/) (2024)
- [Deep Multiple Instance Learning with Distance-Aware Self-Attention](https://mlsanity.com/paper/2305.10552/) (2023)
- [Dual-Query Multiple Instance Learning for Dynamic Meta-Embedding based Tumor Classification](https://mlsanity.com/paper/2307.07482/) (2023)
- [Scribble-based fast weak-supervision and interactive corrections for segmenting whole slide images](https://mlsanity.com/paper/2402.08333/) (2024)
- [Counting Network for Learning from Majority Label](https://mlsanity.com/paper/2403.13370/) (2024)
- [Enhancing Weakly-Supervised Histopathology Image Segmentation with Knowledge Distillation on MIL-Based Pseudo-Labels](https://mlsanity.com/paper/2407.10274/) (2024)
- [EP-SAM: Weakly Supervised Histopathology Segmentation via Enhanced Prompt with Segment Anything](https://mlsanity.com/paper/2410.13621/) (2024)
- [A Novel Confidence Induced Class Activation Mapping for MRI Brain Tumor Segmentation](https://mlsanity.com/paper/2306.05476/) (2023)
- [Leveraging Spatial Context for Positive Pair Sampling in Histopathology Image Representation Learning](https://mlsanity.com/paper/2503.05170/) (2025)
- [DLiPath: A Benchmark for the Comprehensive Assessment of Donor Liver Based on Histopathological Image Dataset](https://mlsanity.com/paper/2506.03185/) (2025)
- [Optimal Transport Driven Asymmetric Image-to-Image Translation for Nuclei Segmentation of Histological Images](https://mlsanity.com/paper/2506.07023/) (2025)
- [SiliCoN: Simultaneous Nuclei Segmentation and Color Normalization of Histological Images](https://mlsanity.com/paper/2506.07028/) (2025)
- [Diffusion Attention Expert Model for Predicting and Semi-automatic Localizing STAS in Lung Cancer Histopathological Images](https://mlsanity.com/paper/2605.16444/) (2026)

## Abstract (arXiv)

Hispathological image segmentation algorithms play a critical role in computer aided diagnosis technology. The development of weakly supervised segmentation algorithm alleviates the problem of medical image annotation that it is time-consuming and labor-intensive. As a subset of weakly supervised learning, Multiple Instance Learning (MIL) has been proven to be effective in segmentation. However, there is a lack of related information between instances in MIL, which limits the further improvement of segmentation performance. In this paper, we propose a novel weakly supervised method for pixel-level segmentation in histopathology images, which introduces Transformer into the MIL framework to capture global or long-range dependencies. The multi-head self-attention in the Transformer establishes the relationship between instances, which solves the shortcoming that instances are independent of each other in MIL. In addition, deep supervision is introduced to overcome the limitation of annotations in weakly supervised methods and make the better utilization of hierarchical information. The state-of-the-art results on the colon cancer dataset demonstrate the superiority of the proposed method compared with other weakly supervised methods. It is worth believing that there is a potential of our approach for various applications in medical images.