# IRS: Incremental Relationship-guided Segmentation for Digital Pathology

arXiv: https://arxiv.org/abs/2505.22855 · HTML: https://mlsanity.com/paper/2505.22855/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 60 references that are in the map.
- First posted on arXiv 28 May 2025.

## Builds on (most cited first, 30 of 31)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [Med3D: Transfer Learning for 3D Medical Image Analysis](https://mlsanity.com/paper/1904.00625/) (2019)
- [Continual learning with hypernetworks](https://mlsanity.com/paper/1906.00695/) (2019)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)
- [EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything](https://mlsanity.com/paper/2312.00863/) (2023)
- [PLOP: Learning without Forgetting for Continual Semantic Segmentation](https://mlsanity.com/paper/2011.11390/) (2020)
- [Incremental Learning Techniques for Semantic Segmentation](https://mlsanity.com/paper/1907.13372/) (2019)
- [Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization](https://mlsanity.com/paper/1802.01569/) (2018)
- [DoDNet: Learning to segment multi-organ and tumors from multiple partially labeled datasets](https://mlsanity.com/paper/2011.10217/) (2020)
- [UniverSeg: Universal Medical Image Segmentation](https://mlsanity.com/paper/2304.06131/) (2023)
- [Deep Hierarchical Semantic Segmentation](https://mlsanity.com/paper/2203.14335/) (2022)
- [Multi-organ Segmentation over Partially Labeled Datasets with Multi-scale Feature Abstraction](https://mlsanity.com/paper/2001.00208/) (2020)
- [Model Zoo: A Growing "Brain" That Learns Continually](https://mlsanity.com/paper/2106.03027/) (2021)
- [An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditions](https://mlsanity.com/paper/2204.08817/) (2022)
- [Continual Learning for Abdominal Multi-Organ and Tumor Segmentation](https://mlsanity.com/paper/2306.00988/) (2023)
- [Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformers](https://mlsanity.com/paper/2204.11432/) (2022)
- [Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learning](https://mlsanity.com/paper/2311.17597/) (2023)
- [A Systematic Collection of Medical Image Datasets for Deep Learning](https://mlsanity.com/paper/2106.12864/) (2021)
- [CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks](https://mlsanity.com/paper/2112.02714/) (2021)
- [CoMFormer: Continual Learning in Semantic and Panoptic Segmentation](https://mlsanity.com/paper/2211.13999/) (2022)
- [Instance-based Vision Transformer for Subtyping of Papillary Renal Cell Carcinoma in Histopathological Image](https://mlsanity.com/paper/2106.12265/) (2021)
- [Iterative annotation to ease neural network training: Specialized machine learning in medical image analysis](https://mlsanity.com/paper/1812.07509/) (2018)
- [Continual Segment: Towards a Single, Unified and Accessible Continual Segmentation Model of 143 Whole-body Organs in CT Scans](https://mlsanity.com/paper/2302.00162/) (2023)
- [Omni-Seg: A Scale-aware Dynamic Network for Renal Pathological Image Segmentation](https://mlsanity.com/paper/2206.13632/) (2022)
- [Low-Rank Mixture-of-Experts for Continual Medical Image Segmentation](https://mlsanity.com/paper/2406.13583/) (2024)
- [PrPSeg: Universal Proposition Learning for Panoramic Renal Pathology Segmentation](https://mlsanity.com/paper/2402.19286/) (2024)
- [MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts](https://mlsanity.com/paper/2310.02000/) (2023)
- [CNN Cascades for Segmenting Whole Slide Images of the Kidney](https://mlsanity.com/paper/1708.00251/) (2017)

## Built on by (most cited first, 2 of 2)

- [Img2ST-Net: Efficient High-Resolution Spatial Omics Prediction from Whole Slide Histology Images via Fully Convolutional Image-to-Image Learning](https://mlsanity.com/paper/2508.14393/) (2025)
- [How Close Are We? Limitations and Progress of AI Models in Banff Lesion Scoring](https://mlsanity.com/paper/2510.27158/) (2025)

## Abstract (arXiv)

Continual learning is rapidly emerging as a key focus in computer vision, aiming to develop AI systems capable of continuous improvement, thereby enhancing their value and practicality in diverse real-world applications. In healthcare, continual learning holds great promise for continuously acquired digital pathology data, which is collected in hospitals on a daily basis. However, panoramic segmentation on digital whole slide images (WSIs) presents significant challenges, as it is often infeasible to obtain comprehensive annotations for all potential objects, spanning from coarse structures (e.g., regions and unit objects) to fine structures (e.g., cells). This results in temporally and partially annotated data, posing a major challenge in developing a holistic segmentation framework. Moreover, an ideal segmentation model should incorporate new phenotypes, unseen diseases, and diverse populations, making this task even more complex. In this paper, we introduce a novel and unified Incremental Relationship-guided Segmentation (IRS) learning scheme to address temporally acquired, partially annotated data while maintaining out-of-distribution (OOD) continual learning capacity in digital pathology. The key innovation of IRS lies in its ability to realize a new spatial-temporal OOD continual learning paradigm by mathematically modeling anatomical relationships between existing and newly introduced classes through a simple incremental universal proposition matrix. Experimental results demonstrate that the IRS method effectively handles the multi-scale nature of pathological segmentation, enabling precise kidney segmentation across various structures (regions, units, and cells) as well as OOD disease lesions at multiple magnifications. This capability significantly enhances domain generalization, making IRS a robust approach for real-world digital pathology applications.