# Leveraging Transfer Learning with Class-Specific Decoders for Laparoscopic Segmentation

arXiv: https://arxiv.org/abs/2607.29509 · HTML: https://mlsanity.com/paper/2607.29509/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 31 Jul 2026.

## Builds on (most cited first, 17 of 17)

- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Transfusion: Understanding Transfer Learning for Medical Imaging](https://mlsanity.com/paper/1902.07208/) (2019)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [CholecSeg8k: A Semantic Segmentation Dataset for Laparoscopic Cholecystectomy Based on Cholec80](https://mlsanity.com/paper/2012.12453/) (2020)
- [Scaling up self-supervised learning for improved surgical foundation models](https://mlsanity.com/paper/2501.09436/) (2025)
- [Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the art](https://mlsanity.com/paper/2304.13014/) (2023)
- [Deep Learning for Surgical Instrument Recognition and Segmentation in Robotic-Assisted Surgeries: A Systematic Review](https://mlsanity.com/paper/2410.07269/) (2024)
- [Jumpstarting Surgical Computer Vision](https://mlsanity.com/paper/2312.05968/) (2023)
- [Data Augmentation for Surgical Scene Segmentation with Anatomy-Aware Diffusion Models](https://mlsanity.com/paper/2410.07753/) (2024)
- [Augmenting Efficient Real-time Surgical Instrument Segmentation in Video with Point Tracking and Segment Anything](https://mlsanity.com/paper/2403.08003/) (2024)
- [SegMatch: A semi-supervised learning method for surgical instrument segmentation](https://mlsanity.com/paper/2308.05232/) (2023)
- [Towards more precise automatic analysis: a comprehensive survey of deep learning-based multi-organ segmentation](https://mlsanity.com/paper/2303.00232/) (2023)
- [One model to use them all: Training a segmentation model with complementary datasets](https://mlsanity.com/paper/2402.19340/) (2024)
- [Critical Assessment of Transfer Learning for Medical Image Segmentation with Fully Convolutional Neural Networks](https://mlsanity.com/paper/2006.00356/) (2020)
- [Strategies to Improve Real-World Applicability of Laparoscopic Anatomy Segmentation Models](https://mlsanity.com/paper/2403.17192/) (2024)
- [MOSMOS: Multi-organ segmentation facilitated by medical report supervision](https://mlsanity.com/paper/2409.02418/) (2024)

## Abstract (arXiv)

Effective multi-organ segmentation in surgical data requires learning the intricate anatomical features and alleviating the challenge of class imbalance, which results from relatively lower proportions of small and limitedly exposed structures. Recent works on laparoscopic multi-organ segmentation focus on learning structure-specific features through class-specific decoder architectures and report favorable results. This work extends the decoder-focused architectures to investigate knowledge sharing in the cross-surgical domain. We utilize two datasets representing different surgical domains, rectal and cholecystectomy surgeries, to explore how surgical conceptual knowledge transfers under partially common anatomical representations. Additionally, we compare the feature adaptation for the encoder and decoder at different training stages to analyse the knowledge adaptation and retention in the network. Our results corroborate previous findings on decoder-specific architectures and demonstrate that the organ-specific decoder model (CEMD), fully fine-tuned after cross-domain pre-training, achieves the highest segmentation performance (62.4% dice) while converging substantially faster than training from scratch. However, we also find that class imbalance in surgical data remains a persistent challenge that transfer learning does not fully resolve for underrepresented anatomical structures.