# Effective semantic segmentation in Cataract Surgery: What matters most?

arXiv: https://arxiv.org/abs/2108.06119 · HTML: https://mlsanity.com/paper/2108.06119/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 13 Aug 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [DeepPhase: Surgical Phase Recognition in CATARACTS Videos](https://mlsanity.com/paper/1807.10565/) (2018)
- [RAUNet: Residual Attention U-Net for Semantic Segmentation of Cataract Surgical Instruments](https://mlsanity.com/paper/1909.10360/) (2019)
- [An Adaptive Sampling Scheme to Efficiently Train Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1709.02764/) (2017)

## Built on by (most cited first, 11 of 11)

- [Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentation](https://mlsanity.com/paper/2203.15251/) (2022)
- [Dynamic Scene Graph Representation for Surgical Video](https://mlsanity.com/paper/2309.14538/) (2023)
- [SANGRIA: Surgical Video Scene Graph Optimization for Surgical Workflow Prediction](https://mlsanity.com/paper/2407.20214/) (2024)
- [Multi-scale and Cross-scale Contrastive Learning for Semantic Segmentation](https://mlsanity.com/paper/2203.13409/) (2022)
- [Joint Class-Affinity Loss Correction for Robust Medical Image Segmentation with Noisy Labels](https://mlsanity.com/paper/2206.07994/) (2022)
- [DeepPyramid: Enabling Pyramid View and Deformable Pyramid Reception for Semantic Segmentation in Cataract Surgery Videos](https://mlsanity.com/paper/2207.01453/) (2022)
- [Deep learning approaches to surgical video segmentation and object detection: A Scoping Review](https://mlsanity.com/paper/2502.16459/) (2025)
- [Semi-Supervised Medical Image Segmentation with Co-Distribution Alignment](https://mlsanity.com/paper/2307.12630/) (2023)
- [SURGIVID: Annotation-Efficient Surgical Video Object Discovery](https://mlsanity.com/paper/2409.07801/) (2024)
- [DeepPyram: Enabling Pyramid View and Deformable Pyramid Reception for Semantic Segmentation in Cataract Surgery Videos](https://mlsanity.com/paper/2109.05352/) (2021)
- [CataractCompDetect: Intraoperative Complication Detection in Cataract Surgery](https://mlsanity.com/paper/2511.18968/) (2025)

## Abstract (arXiv)

Our work proposes neural network design choices that set the state-of-the-art on a challenging public benchmark on cataract surgery, CaDIS. Our methodology achieves strong performance across three semantic segmentation tasks with increasingly granular surgical tool class sets by effectively handling class imbalance, an inherent challenge in any surgical video. We consider and evaluate two conceptually simple data oversampling methods as well as different loss functions. We show significant performance gains across network architectures and tasks especially on the rarest tool classes, thereby presenting an approach for achieving high performance when imbalanced granular datasets are considered. Our code and trained models are available at https://github.com/RViMLab/MICCAI2021_Cataract_semantic_segmentation and qualitative results on unseen surgical video can be found at https://youtu.be/twVIPUj1WZM.