# Unlocking Positive Transfer in Incrementally Learning Surgical Instruments: A Self-reflection Hierarchical Prompt Framework

arXiv: https://arxiv.org/abs/2604.02877 · HTML: https://mlsanity.com/paper/2604.02877/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 3 Apr 2026.

## Builds on (most cited first, 28 of 28)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Learning to Prompt for Continual Learning](https://mlsanity.com/paper/2112.08654/) (2021)
- [DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning](https://mlsanity.com/paper/2204.04799/) (2022)
- [S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning](https://mlsanity.com/paper/2207.12819/) (2022)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)
- [PLOP: Learning without Forgetting for Continual Semantic Segmentation](https://mlsanity.com/paper/2011.11390/) (2020)
- [Incremental Learning Techniques for Semantic Segmentation](https://mlsanity.com/paper/1907.13372/) (2019)
- [Don't forget, there is more than forgetting: new metrics for Continual Learning](https://mlsanity.com/paper/1810.13166/) (2018)
- [CholecSeg8k: A Semantic Segmentation Dataset for Laparoscopic Cholecystectomy Based on Cholec80](https://mlsanity.com/paper/2012.12453/) (2020)
- [Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations](https://mlsanity.com/paper/2103.06342/) (2021)
- [AFEC: Active Forgetting of Negative Transfer in Continual Learning](https://mlsanity.com/paper/2110.12187/) (2021)
- [Representation Compensation Networks for Continual Semantic Segmentation](https://mlsanity.com/paper/2203.05402/) (2022)
- [ISINet: An Instance-Based Approach for Surgical Instrument Segmentation](https://mlsanity.com/paper/2007.05533/) (2020)
- [m2caiSeg: Semantic Segmentation of Laparoscopic Images using Convolutional Neural Networks](https://mlsanity.com/paper/2008.10134/) (2020)
- [Multi-Domain Incremental Learning for Semantic Segmentation](https://mlsanity.com/paper/2110.12205/) (2021)
- [Prompt Tuning for Parameter-efficient Medical Image Segmentation](https://mlsanity.com/paper/2211.09233/) (2022)
- [Learning Incrementally to Segment Multiple Organs in a CT Image](https://mlsanity.com/paper/2203.02100/) (2022)
- [CoinSeg: Contrast Inter- and Intra- Class Representations for Incremental Segmentation](https://mlsanity.com/paper/2310.06368/) (2023)
- [Domain-incremental Cardiac Image Segmentation with Style-oriented Replay and Domain-sensitive Feature Whitening](https://mlsanity.com/paper/2211.04862/) (2022)
- [CAPrompt: Cyclic Prompt Aggregation for Pre-Trained Model Based Class Incremental Learning](https://mlsanity.com/paper/2412.08929/) (2024)
- [Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgery](https://mlsanity.com/paper/2402.05860/) (2024)
- [CODE-CL: Conceptor-Based Gradient Projection for Deep Continual Learning](https://mlsanity.com/paper/2411.15235/) (2024)
- [Memory-Efficient Prompt Tuning for Incremental Histopathology Classification](https://mlsanity.com/paper/2401.11674/) (2024)
- [The Surprising Positive Knowledge Transfer in Continual 3D Object Shape Reconstruction](https://mlsanity.com/paper/2101.07295/) (2021)
- [PEARL: Input-Agnostic Prompt Enhancement with Negative Feedback Regulation for Class-Incremental Learning](https://mlsanity.com/paper/2412.10900/) (2024)

## Built on by (most cited first, 1 of 1)

- [Hierarchical Prototype-Memory Adaptation of SAM for Surgical Instrument Segmentation](https://mlsanity.com/paper/2608.24541/) (2026)

## Abstract (arXiv)

To continuously enhance model adaptability in surgical video scene parsing, recent studies incrementally update it to progressively learn to segment an increasing number of surgical instruments over time. However, prior works constantly overlooked the potential of positive forward knowledge transfer, i.e., how past knowledge could help learn new classes, and positive backward knowledge transfer, i.e., how learning new classes could help refine past knowledge. In this paper, we propose a self-reflection hierarchical prompt framework that unlocks the power of positive forward and backward knowledge transfer in class incremental segmentation, aiming to proficiently learn new instruments, improve existing skills of regular instruments, and avoid catastrophic forgetting of old instruments. Our framework is built on a frozen, pre-trained model that adaptively appends instrument-aware prompts for new classes throughout training episodes. To enable positive forward knowledge transfer, we organize instrument prompts into a hierarchical prompt parsing tree with the instrument-shared prompt partition as the root node, n-part-shared prompt partitions as intermediate nodes and instrument-distinct prompt partitions as leaf nodes, to expose the reusable historical knowledge for new classes to simplify their learning. Conversely, to encourage positive backward knowledge transfer, we conduct self-reflection refining on existing knowledge by directed-weighted graph propagation, examining the knowledge associations recorded in the tree to improve its representativeness without causing catastrophic forgetting. Our framework is applicable to both CNN-based models and advanced transformer-based foundation models, yielding more than 5% and 11% improvements over the competing methods on two public benchmarks respectively.