mlsanityData to 14 Oct 2026

Rethinking Exemplars for Continual Semantic Segmentation in Endoscopy Scenes: Entropy-based Mini-Batch Pseudo-Replay

arXiv 2308.14100 ↗First posted 27 Aug 2023cs.CV cs.AIcited by 20

Not on the map. Why?

Abstract (arXiv)

Endoscopy is a widely used technique for the early detection of diseases or robotic-assisted minimally invasive surgery (RMIS). Numerous deep learning (DL)-based research works have been developed for automated diagnosis or processing of endoscopic view. However, existing DL models may suffer from catastrophic forgetting. When new target classes are introduced over time or cross institutions, the performance of old classes may suffer severe degradation. More seriously, data privacy and storage issues may lead to the unavailability of old data when updating the model. Therefore, it is necessary to develop a continual learning (CL) methodology to solve the problem of catastrophic forgetting in endoscopic image segmentation. To tackle this, we propose a Endoscopy Continual Semantic Segmentation (EndoCSS) framework that does not involve the storage and privacy issues of exemplar data. The framework includes a mini-batch pseudo-replay (MB-PR) mechanism and a self-adaptive noisy cross-entropy (SAN-CE) loss. The MB-PR strategy circumvents privacy and storage issues by generating pseudo-replay images through a generative model. Meanwhile, the MB-PR strategy can also correct the model deviation to the replay data and current training data, which is aroused by the significant difference in the amount of current and replay images. Therefore, the model can perform effective representation learning on both new and old tasks. SAN-CE loss can help model fitting by adjusting the model's output logits, and also improve the robustness of training. Extensive continual semantic segmentation (CSS) experiments on public datasets demonstrate that our method can robustly and effectively address the catastrophic forgetting brought by class increment in endoscopy scenes. The results show that our framework holds excellent potential for real-world deployment in a streaming learning manner.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 56 of its 75 references that are in the map. First posted on arXiv 27 Aug 2023.

Builds on

56 of 75 references.

References · 56

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2022Generative Adversarial Networkscited by 5,627
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  7. 2016Learning without Forgettingcited by 2,496
  8. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  9. 2015Variational Inference with Normalizing Flowscited by 2,170
  10. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  11. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  12. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 742
38 more
  1. 2018ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentationcited by 623
  2. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  3. 2022Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labelscited by 218
  4. 2020Modeling the Background for Incremental Learning in Semantic Segmentationcited by 190
  5. 2020PLOP: Learning without Forgetting for Continual Semantic Segmentationcited by 177
  6. 20192017 Robotic Instrument Segmentation Challengecited by 168
  7. 2019Incremental Learning Techniques for Semantic Segmentationcited by 141
  8. 20202018 Robotic Scene Segmentation Challengecited by 124
  9. 2019Synthetic Data for Deep Learningcited by 113
  10. 2019Continual Learning for Robotics: Definition, Framework, Learning Strategies, Opportunities and Challengescited by 93
  11. 2021RECALL: Replay-based Continual Learning in Semantic Segmentationcited by 83
  12. 2022Representation Compensation Networks for Continual Semantic Segmentationcited by 72
  13. 2021SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learningcited by 71
  14. 2020ISINet: An Instance-Based Approach for Surgical Instrument Segmentationcited by 67
  15. 2021Drop the GAN: In Defense of Patches Nearest Neighbors as Single Image Generative Modelscited by 51
  16. 2018A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocolscited by 49
  17. 2022SinDiffusion: Learning a Diffusion Model from a Single Natural Imagecited by 42
  18. 2020Learning and Reasoning with the Graph Structure Representation in Robotic Surgerycited by 29
  19. 2022Global-Reasoned Multi-Task Learning Model for Surgical Scene Understandingcited by 29
  20. 2020What is Wrong with Continual Learning in Medical Image Segmentation?cited by 25
  21. 2018Towards continual learning in medical imagingcited by 24
  22. 2017Noisy Softmax: Improving the Generalization Ability of DCNN via Postponing the Early Softmax Saturationcited by 21
  23. 2020Importance Driven Continual Learning for Segmentation Across Domainscited by 21
  24. 2023Towards Generalizable Diabetic Retinopathy Grading in Unseen Domainscited by 21
  25. 2022Learning Robust Representation for Joint Grading of Ophthalmic Diseases via Adaptive Curriculum and Feature Disentanglementcited by 20
  26. 2019Automatic Colon Polyp Detection using Region based Deep CNN and Post Learning Approachescited by 16
  27. 2022FisherMatch: Semi-Supervised Rotation Regression via Entropy-based Filteringcited by 16
  28. 2023Revisiting Distillation for Continual Learning on Visual Question Localized-Answering in Robotic Surgerycited by 15
  29. 2020Endoscopy disease detection challenge 2020cited by 14
  30. 2021Adversarial Continual Learning for Multi-Domain Hippocampal Segmentationcited by 14
  31. 2021Addressing catastrophic forgetting for medical domain expansioncited by 12
  32. 2023Intuitive Surgical SurgToolLoc and SurgVU Challenges Results: 2022-2025cited by 12
  33. 2019Towards continuous learning for glioma segmentation with elastic weight consolidationcited by 11
  34. 2020Batch-level Experience Replay with Review for Continual Learningcited by 10
  35. 2022Two-stage Contextual Transformer-based Convolutional Neural Network for Airway Extraction from CT Imagescited by 10
  36. 2023Deep Reinforcement Learning-Based Control for Stomach Coverage Scanning of Wireless Capsule Endoscopycited by 9
  37. 2021Training Generative Adversarial Networks in One Stagecited by 6
  38. 2021The Influence of Age and Gender Information on the Diagnosis of Diabetic Retinopathy: Based on Neural Networkscited by 6

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 13; 2025: 3; 2026: 2.

20 papers.

Sort:

By area:

  1. 2023Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospectscited by 22
  2. 2023A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Applicationcited by 21
  3. 2024SAM 2 in Robotic Surgery: An Empirical Evaluation for Robustness and Generalization in Surgical Video Segmentationcited by 21
  4. 2024kNN-CLIP: Retrieval Enables Training-Free Segmentation on Continually Expanding Large Vocabulariescited by 20
  5. 2024Surgical-VQLA++: Adversarial Contrastive Learning for Calibrated Robust Visual Question-Localized Answering in Robotic Surgerycited by 17
  6. 2024LighTDiff: Surgical Endoscopic Image Low-Light Enhancement with T-Diffusioncited by 12
  7. 2024EndoOOD: Uncertainty-aware Out-of-distribution Detection in Capsule Endoscopy Diagnosiscited by 9
  8. 2024Learning to Adapt Foundation Model DINOv2 for Capsule Endoscopy Diagnosiscited by 8
  9. 2024EndoUIC: Promptable Diffusion Transformer for Unified Illumination Correction in Capsule Endoscopycited by 8
  10. 2024OSSAR: Towards Open-Set Surgical Activity Recognition in Robot-assisted Surgerycited by 7
  11. 2024Continual Panoptic Perception: Towards Multi-modal Incremental Interpretation of Remote Sensing Imagescited by 4
  12. 2024Transferring Knowledge from High-Quality to Low-Quality MRI for Adult Glioma Diagnosiscited by 2
8 more
  1. 2024CFSSeg: Closed-Form Solution for Class-Incremental Semantic Segmentation of 2D Images and 3D Point Cloudscited by 2
  2. 2024TSUBF-Net: Trans-Spatial UNet-like Network with Bi-direction Fusion for Segmentation of Adenoid Hypertrophy in CTcited by 1
  3. 2025EndoARSS: Adapting Spatially-Aware Foundation Model for Efficient Activity Recognition and Semantic Segmentation in Endoscopic Surgerycited by 1
  4. 2025CapsDT: Diffusion-Transformer for Capsule Robot Manipulationcited by 1
  5. 2025Continual Alignment for SAM: Rethinking Foundation Models for Medical Image Segmentation in Continual Learningcited by 1
  6. 2024V2-SfMLearner: Learning Monocular Depth and Ego-motion for Multimodal Wireless Capsule Endoscopycited by 0
  7. 2026Evolving Without Ending: Unifying Multimodal Incremental Learning for Continual Panoptic Perceptioncited by 0
  8. 2026Restoring Without Forgetting: Continual Learning Across Image Degradationscited by 0