# Class-Incremental Domain Adaptation with Smoothing and Calibration for Surgical Report Generation arXiv: https://arxiv.org/abs/2107.11091 ยท HTML: https://mlsanity.com/paper/2107.11091/ ## Key facts - Cited by 18 papers in the map (data through 14 Oct 2026). - Builds on 20 of its 26 references that are in the map. - First posted on arXiv 23 Jul 2021. ## Builds on (most cited first, 20 of 20) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015) - [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015) - [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020) - [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014) - [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021) - [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019) - [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018) - [Measuring Calibration in Deep Learning](https://mlsanity.com/paper/1904.01685/) (2019) - [Meshed-Memory Transformer for Image Captioning](https://mlsanity.com/paper/1912.08226/) (2019) - [Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning](https://mlsanity.com/paper/2011.01403/) (2020) - [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020) - [X-Linear Attention Networks for Image Captioning](https://mlsanity.com/paper/2003.14080/) (2020) - [2018 Robotic Scene Segmentation Challenge](https://mlsanity.com/paper/2001.11190/) (2020) - [Class-Incremental Domain Adaptation](https://mlsanity.com/paper/2008.01389/) (2020) - [Learning and Reasoning with the Graph Structure Representation in Robotic Surgery](https://mlsanity.com/paper/2007.03357/) (2020) - [Endo-Sim2Real: Consistency learning-based domain adaptation for instrument segmentation](https://mlsanity.com/paper/2007.11514/) (2020) - [Learning Domain Adaptation with Model Calibration for Surgical Report Generation in Robotic Surgery](https://mlsanity.com/paper/2103.17120/) (2021) - [Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge](https://mlsanity.com/paper/2102.13644/) (2021) ## Built on by (most cited first, 18 of 18) - [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023) - [Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lectures](https://mlsanity.com/paper/2307.15220/) (2023) - [Prototype-Guided Continual Adaptation for Class-Incremental Unsupervised Domain Adaptation](https://mlsanity.com/paper/2207.10856/) (2022) - [General Incremental Learning with Domain-aware Categorical Representations](https://mlsanity.com/paper/2204.04078/) (2022) - [Rethinking Surgical Instrument Segmentation: A Background Image Can Be All You Need](https://mlsanity.com/paper/2206.11804/) (2022) - [S2ME: Spatial-Spectral Mutual Teaching and Ensemble Learning for Scribble-supervised Polyp Segmentation](https://mlsanity.com/paper/2306.00451/) (2023) - [Dynamic Interactive Relation Capturing via Scene Graph Learning for Robotic Surgical Report Generation](https://mlsanity.com/paper/2306.02651/) (2023) - [A Systematic Review of Generalization Research in Medical Image Classification](https://mlsanity.com/paper/2403.12167/) (2024) - [Curriculum-Based Augmented Fourier Domain Adaptation for Robust Medical Image Segmentation](https://mlsanity.com/paper/2306.03511/) (2023) - [Taxonomy Adaptive Cross-Domain Adaptation in Medical Imaging via Optimization Trajectory Distillation](https://mlsanity.com/paper/2307.14709/) (2023) - [Beyond Role-Based Surgical Domain Modeling: Generalizable Re-Identification in the Operating Room](https://mlsanity.com/paper/2503.13028/) (2025) - [Rethinking Surgical Captioning: End-to-End Window-Based MLP Transformer Using Patches](https://mlsanity.com/paper/2207.00113/) (2022) - [Online Continual Learning via Logit Adjusted Softmax](https://mlsanity.com/paper/2311.06460/) (2023) - [Multitask Learning in Minimally Invasive Surgical Vision: A Review](https://mlsanity.com/paper/2401.08256/) (2024) - [Task-Aware Asynchronous Multi-Task Model with Class Incremental Contrastive Learning for Surgical Scene Understanding](https://mlsanity.com/paper/2211.15327/) (2022) - [A comprehensive survey on recent deep learning-based methods applied to surgical data](https://mlsanity.com/paper/2209.01435/) (2022) - [Generalizing Surgical Instruments Segmentation to Unseen Domains with One-to-Many Synthesis](https://mlsanity.com/paper/2306.16285/) (2023) - [Confidence-Aware Paced-Curriculum Learning by Label Smoothing for Surgical Scene Understanding](https://mlsanity.com/paper/2212.11511/) (2022) ## Abstract (arXiv) Generating surgical reports aimed at surgical scene understanding in robot-assisted surgery can contribute to documenting entry tasks and post-operative analysis. Despite the impressive outcome, the deep learning model degrades the performance when applied to different domains encountering domain shifts. In addition, there are new instruments and variations in surgical tissues appeared in robotic surgery. In this work, we propose class-incremental domain adaptation (CIDA) with a multi-layer transformer-based model to tackle the new classes and domain shift in the target domain to generate surgical reports during robotic surgery. To adapt incremental classes and extract domain invariant features, a class-incremental (CI) learning method with supervised contrastive (SupCon) loss is incorporated with a feature extractor. To generate caption from the extracted feature, curriculum by one-dimensional gaussian smoothing (CBS) is integrated with a multi-layer transformer-based caption prediction model. CBS smoothes the features embedding using anti-aliasing and helps the model to learn domain invariant features. We also adopt label smoothing (LS) to calibrate prediction probability and obtain better feature representation with both feature extractor and captioning model. The proposed techniques are empirically evaluated by using the datasets of two surgical domains, such as nephrectomy operations and transoral robotic surgery. We observe that domain invariant feature learning and the well-calibrated network improves the surgical report generation performance in both source and target domain under domain shift and unseen classes in the manners of one-shot and few-shot learning. The code is publicly available at https://github.com/XuMengyaAmy/CIDACaptioning.