# The Endoscapes Dataset for Surgical Scene Segmentation, Object Detection, and Critical View of Safety Assessment: Official Splits and Benchmark

arXiv: https://arxiv.org/abs/2312.12429 · HTML: https://mlsanity.com/paper/2312.12429/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 19 Dec 2023.

## Builds on (most cited first, 14 of 14)

- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [Videos as Space-Time Region Graphs](https://mlsanity.com/paper/1806.01810/) (2018)
- [Surgical Data Science: Enabling Next-Generation Surgery](https://mlsanity.com/paper/1701.06482/) (2017)
- [CholecSeg8k: A Semantic Segmentation Dataset for Laparoscopic Cholecystectomy Based on Cholec80](https://mlsanity.com/paper/2012.12453/) (2020)
- [Surgical Data Science -- from Concepts toward Clinical Translation](https://mlsanity.com/paper/2011.02284/) (2020)
- [Dissecting Self-Supervised Learning Methods for Surgical Computer Vision](https://mlsanity.com/paper/2207.00449/) (2022)
- [DeepPhase: Surgical Phase Recognition in CATARACTS Videos](https://mlsanity.com/paper/1807.10565/) (2018)
- [Latent Graph Representations for Critical View of Safety Assessment](https://mlsanity.com/paper/2212.04155/) (2022)
- [Data Splits and Metrics for Method Benchmarking on Surgical Action Triplet Datasets](https://mlsanity.com/paper/2204.05235/) (2022)
- [Encoding Surgical Videos as Latent Spatiotemporal Graphs for Object and Anatomy-Driven Reasoning](https://mlsanity.com/paper/2312.06829/) (2023)
- [Concept Graph Neural Networks for Surgical Video Understanding](https://mlsanity.com/paper/2202.13402/) (2022)
- [Surgical data science for safe cholecystectomy: a protocol for segmentation of hepatocystic anatomy and assessment of the critical view of safety](https://mlsanity.com/paper/2106.10916/) (2021)
- [Temporally Constrained Neural Networks (TCNN): A framework for semi-supervised video semantic segmentation](https://mlsanity.com/paper/2112.13815/) (2021)

## Built on by (most cited first, 21 of 21)

- [The SAGES Critical View of Safety Challenge: A Global Benchmark for AI-Assisted Surgical Quality Assessment](https://mlsanity.com/paper/2509.17100/) (2025)
- [Multitask Learning in Minimally Invasive Surgical Vision: A Review](https://mlsanity.com/paper/2401.08256/) (2024)
- [Systematic Evaluation and Guidelines for Segment Anything Model in Surgical Video Analysis](https://mlsanity.com/paper/2501.00525/) (2024)
- [Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence](https://mlsanity.com/paper/2504.02799/) (2025)
- [Artificial Intelligence for Biomedical Video Generation](https://mlsanity.com/paper/2411.07619/) (2024)
- [Multi-modal Representations for Fine-grained Multi-label Critical View of Safety Recognition](https://mlsanity.com/paper/2507.05007/) (2025)
- [SurgSigma: A Spectrum of Large-Scale Multimodal Data and Foundation Models for Surgical Intelligence](https://mlsanity.com/paper/2603.16822/) (2026)
- [SurgAtlas: A Large-Scale Surgical Video-Language Dataset with 2,391 Hours of Open and Minimally Invasive Surgery](https://mlsanity.com/paper/2606.25905/) (2026)
- [Optimizing Latent Graph Representations of Surgical Scenes for Zero-Shot Domain Transfer](https://mlsanity.com/paper/2403.06953/) (2024)
- [Temporal Propagation of Asymmetric Feature Pyramid for Surgical Scene Segmentation](https://mlsanity.com/paper/2504.13440/) (2025)
- [LapFM: A Laparoscopic Segmentation Foundation Model via Hierarchical Concept Evolving Pre-training](https://mlsanity.com/paper/2512.08439/) (2025)
- [Scaling Video Pretraining for Surgical Foundation Models](https://mlsanity.com/paper/2603.29966/) (2026)
- [Surgical Anatomy Recognition with Context Learning using Foundation Representations](https://mlsanity.com/paper/2606.22124/) (2026)
- [Robust Noisy Pseudo-label Learning for Semi-supervised Medical Image Segmentation Using Diffusion Model](https://mlsanity.com/paper/2507.16429/) (2025)
- [Graph Neural Networks for Surgical Scene Segmentation](https://mlsanity.com/paper/2511.16430/) (2025)
- [Sum-of-Checks: Structured Reasoning for Surgical Safety with Large Vision-Language Models](https://mlsanity.com/paper/2604.22156/) (2026)
- [AI for Quality Assurance in the Operating Room](https://mlsanity.com/paper/2606.30657/) (2026)
- [Stitch-Inferencer: Enhance Endoscopic Video Segmentation and Tracking via Panoramic Reconstruction](https://mlsanity.com/paper/2607.14968/) (2026)
- [SetPlanner: A Lightweight Plug-in Point-Set Planner for Frozen SAM](https://mlsanity.com/paper/2609.18037/) (2026)
- [CasCVS-Net: A Staged Multi-Task Cascade for Critical View of Safety Assessment](https://mlsanity.com/paper/2609.27681/) (2026)
- [Structured Reasoning Agentic Framework for Interpretable Critical View of Safety Assessment](https://mlsanity.com/paper/2609.31524/) (2026)

## Abstract (arXiv)

This technical report provides a detailed overview of Endoscapes, a dataset of laparoscopic cholecystectomy (LC) videos with highly intricate annotations targeted at automated assessment of the Critical View of Safety (CVS). Endoscapes comprises 201 LC videos with frames annotated sparsely but regularly with segmentation masks, bounding boxes, and CVS assessment by three different clinical experts. Altogether, there are 11090 frames annotated with CVS and 1933 frames annotated with tool and anatomy bounding boxes from the 201 videos, as well as an additional 422 frames from 50 of the 201 videos annotated with tool and anatomy segmentation masks. In this report, we provide detailed dataset statistics (size, class distribution, dataset splits, etc.) and a comprehensive performance benchmark for instance segmentation, object detection, and CVS prediction. The dataset and model checkpoints are publically available at https://github.com/CAMMA-public/Endoscapes.