# Semi-supervised Pathological Image Segmentation via Cross Distillation of Multiple Attentions

arXiv: https://arxiv.org/abs/2305.18830 · HTML: https://mlsanity.com/paper/2305.18830/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 25 references that are in the map.
- First posted on arXiv 30 May 2023.

## Builds on (most cited first, 16 of 16)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/1811.12833/) (2018)
- [Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision](https://mlsanity.com/paper/2106.01226/) (2021)
- [Decoupled Knowledge Distillation](https://mlsanity.com/paper/2203.08679/) (2022)
- [Uncertainty-aware Self-ensembling Model for Semi-supervised 3D Left Atrium Segmentation](https://mlsanity.com/paper/1907.07034/) (2019)
- [Semi-Supervised Semantic Segmentation with Cross-Consistency Training](https://mlsanity.com/paper/2003.09005/) (2020)
- [R-Drop: Regularized Dropout for Neural Networks](https://mlsanity.com/paper/2106.14448/) (2021)
- [Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images](https://mlsanity.com/paper/2004.14133/) (2020)
- [Mutual Consistency Learning for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2109.09960/) (2021)
- [Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformer](https://mlsanity.com/paper/2112.04894/) (2021)
- [Recalibrating Fully Convolutional Networks with Spatial and Channel 'Squeeze & Excitation' Blocks](https://mlsanity.com/paper/1808.08127/) (2018)
- [Pairwise Relation Learning for Semi-supervised Gland Segmentation](https://mlsanity.com/paper/2008.02699/) (2020)
- [Dual Adaptive Pyramid Network for Cross-Stain Histopathology Image Segmentation](https://mlsanity.com/paper/1909.11524/) (2019)

## Built on by (most cited first, 8 of 8)

- [Learnable Prompting SAM-induced Knowledge Distillation for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2412.13742/) (2024)
- [Dual-scale Enhanced and Cross-generative Consistency Learning for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2312.16039/) (2023)
- [Image Distillation for Safe Data Sharing in Histopathology](https://mlsanity.com/paper/2406.13536/) (2024)
- [Uncertainty-aware Cross-training for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2508.09014/) (2025)
- [MetaSSL: A General Heterogeneous Loss for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2509.01144/) (2025)
- [Striving for Simplicity: Simple Yet Effective Prior-Aware Pseudo-Labeling for Semi-Supervised Ultrasound Image Segmentation](https://mlsanity.com/paper/2503.13987/) (2025)
- [Cross-pyramid consistency regularization for semi-supervised medical image segmentation](https://mlsanity.com/paper/2511.08435/) (2025)
- [SemiGDA: Generative Dual-distribution Alignment for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2604.23274/) (2026)

## Abstract (arXiv)

Segmentation of pathological images is a crucial step for accurate cancer diagnosis. However, acquiring dense annotations of such images for training is labor-intensive and time-consuming. To address this issue, Semi-Supervised Learning (SSL) has the potential for reducing the annotation cost, but it is challenged by a large number of unlabeled training images. In this paper, we propose a novel SSL method based on Cross Distillation of Multiple Attentions (CDMA) to effectively leverage unlabeled images. Firstly, we propose a Multi-attention Tri-branch Network (MTNet) that consists of an encoder and a three-branch decoder, with each branch using a different attention mechanism that calibrates features in different aspects to generate diverse outputs. Secondly, we introduce Cross Decoder Knowledge Distillation (CDKD) between the three decoder branches, allowing them to learn from each other's soft labels to mitigate the negative impact of incorrect pseudo labels in training. Additionally, uncertainty minimization is applied to the average prediction of the three branches, which further regularizes predictions on unlabeled images and encourages inter-branch consistency. Our proposed CDMA was compared with eight state-of-the-art SSL methods on the public DigestPath dataset, and the experimental results showed that our method outperforms the other approaches under different annotation ratios. The code is available at hrefhttps://github.com/HiLab-git/CDMAhttps://github.com/HiLab-git/CDMA.