# DeepPyramid: Enabling Pyramid View and Deformable Pyramid Reception for Semantic Segmentation in Cataract Surgery Videos

arXiv: https://arxiv.org/abs/2207.01453 · HTML: https://mlsanity.com/paper/2207.01453/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 58 references that are in the map.
- First posted on arXiv 4 Jul 2022.

## Builds on (most cited first, 30 of 36)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation](https://mlsanity.com/paper/1912.05074/) (2019)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [Deeply-Supervised Nets](https://mlsanity.com/paper/1409.5185/) (2014)
- [CE-Net: Context Encoder Network for 2D Medical Image Segmentation](https://mlsanity.com/paper/1903.02740/) (2019)
- [MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation](https://mlsanity.com/paper/1902.04049/) (2019)
- [Attention to Scale: Scale-aware Semantic Image Segmentation](https://mlsanity.com/paper/1511.03339/) (2015)
- [Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagation](https://mlsanity.com/paper/1912.04573/) (2019)
- [Recalibrating Fully Convolutional Networks with Spatial and Channel 'Squeeze & Excitation' Blocks](https://mlsanity.com/paper/1808.08127/) (2018)
- [Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes](https://mlsanity.com/paper/1611.08323/) (2016)
- [Incorporating Temporal Prior from Motion Flow for Instrument Segmentation in Minimally Invasive Surgery Video](https://mlsanity.com/paper/1907.07899/) (2019)
- [DeepPhase: Surgical Phase Recognition in CATARACTS Videos](https://mlsanity.com/paper/1807.10565/) (2018)
- [3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes](https://mlsanity.com/paper/1607.00582/) (2016)
- [Deeply-Supervised CNN for Prostate Segmentation](https://mlsanity.com/paper/1703.07523/) (2017)
- [RAUNet: Residual Attention U-Net for Semantic Segmentation of Cataract Surgical Instruments](https://mlsanity.com/paper/1909.10360/) (2019)
- [CaDIS: Cataract Dataset for Image Segmentation](https://mlsanity.com/paper/1906.11586/) (2019)
- [Relevance Detection in Cataract Surgery Videos by Spatio-Temporal Action Localization](https://mlsanity.com/paper/2104.14280/) (2021)
- [CataNet: Predicting remaining cataract surgery duration](https://mlsanity.com/paper/2106.11048/) (2021)
- [LensID: A CNN-RNN-Based Framework Towards Lens Irregularity Detection in Cataract Surgery Videos](https://mlsanity.com/paper/2107.00875/) (2021)

## Built on by (most cited first, 7 of 7)

- [Domain Adaptation for Medical Image Segmentation using Transformation-Invariant Self-Training](https://mlsanity.com/paper/2307.16660/) (2023)
- [Cataract-1K: Cataract Surgery Dataset for Scene Segmentation, Phase Recognition, and Irregularity Detection](https://mlsanity.com/paper/2312.06295/) (2023)
- [SegSTRONG-C: Segmenting Surgical Tools Robustly On Non-adversarial Generated Corruptions -- An EndoVis'24 Challenge](https://mlsanity.com/paper/2407.11906/) (2024)
- [GynSurg: A Comprehensive Gynecology Laparoscopic Surgery Dataset](https://mlsanity.com/paper/2506.11356/) (2025)
- [Unified Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2311.13254/) (2023)
- [Predicting Postoperative Intraocular Lens Dislocation in Cataract Surgery via Deep Learning](https://mlsanity.com/paper/2312.03401/) (2023)
- [WetCat: Enabling Automated Skill Assessment in Wet-Lab Cataract Surgery Videos](https://mlsanity.com/paper/2506.08896/) (2025)

## Abstract (arXiv)

Semantic segmentation in cataract surgery has a wide range of applications contributing to surgical outcome enhancement and clinical risk reduction. However, the varying issues in segmenting the different relevant structures in these surgeries make the designation of a unique network quite challenging. This paper proposes a semantic segmentation network, termed DeepPyramid, that can deal with these challenges using three novelties: (1) a Pyramid View Fusion module which provides a varying-angle global view of the surrounding region centering at each pixel position in the input convolutional feature map; (2) a Deformable Pyramid Reception module which enables a wide deformable receptive field that can adapt to geometric transformations in the object of interest; and (3) a dedicated Pyramid Loss that adaptively supervises multi-scale semantic feature maps. Combined, we show that these modules can effectively boost semantic segmentation performance, especially in the case of transparency, deformability, scalability, and blunt edges in objects. We demonstrate that our approach performs at a state-of-the-art level and outperforms a number of existing methods with a large margin (3.66% overall improvement in intersection over union compared to the best rival approach).