# Data Splits and Metrics for Method Benchmarking on Surgical Action Triplet Datasets

arXiv: https://arxiv.org/abs/2204.05235 · HTML: https://mlsanity.com/paper/2204.05235/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 11 Apr 2022.

## Builds on (most cited first, 14 of 14)

- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videos](https://mlsanity.com/paper/2109.03223/) (2021)
- [Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformer](https://mlsanity.com/paper/2103.09712/) (2021)
- [Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets](https://mlsanity.com/paper/2007.05405/) (2020)
- [Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks](https://mlsanity.com/paper/1802.08774/) (2018)
- [CholecTriplet2021: A benchmark challenge for surgical action triplet recognition](https://mlsanity.com/paper/2204.04746/) (2022)
- [The SARAS Endoscopic Surgeon Action Detection (ESAD) dataset: Challenges and methods](https://mlsanity.com/paper/2104.03178/) (2021)
- [Weakly Supervised Convolutional LSTM Approach for Tool Tracking in Laparoscopic Videos](https://mlsanity.com/paper/1812.01366/) (2018)
- [ToolNet: Holistically-Nested Real-Time Segmentation of Robotic Surgical Tools](https://mlsanity.com/paper/1706.08126/) (2017)
- [Weakly-Supervised Learning for Tool Localization in Laparoscopic Videos](https://mlsanity.com/paper/1806.05573/) (2018)
- [Robust Medical Instrument Segmentation Challenge 2019](https://mlsanity.com/paper/2003.10299/) (2020)
- [CaDIS: Cataract Dataset for Image Segmentation](https://mlsanity.com/paper/1906.11586/) (2019)
- [Temporal coherence-based self-supervised learning for laparoscopic workflow analysis](https://mlsanity.com/paper/1806.06811/) (2018)

## Built on by (most cited first, 23 of 23)

- [Pixel-Wise Recognition for Holistic Surgical Scene Understanding](https://mlsanity.com/paper/2401.11174/) (2024)
- [Rendezvous in Time: An Attention-based Temporal Fusion approach for Surgical Triplet Recognition](https://mlsanity.com/paper/2211.16963/) (2022)
- [CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection](https://mlsanity.com/paper/2302.06294/) (2023)
- [Surgical Action Triplet Detection by Mixed Supervised Learning of Instrument-Tissue Interactions](https://mlsanity.com/paper/2307.09548/) (2023)
- [The Endoscapes Dataset for Surgical Scene Segmentation, Object Detection, and Critical View of Safety Assessment: Official Splits and Benchmark](https://mlsanity.com/paper/2312.12429/) (2023)
- [GP-VLS: A general-purpose vision language model for surgery](https://mlsanity.com/paper/2407.19305/) (2024)
- [Large-scale Self-supervised Video Foundation Model for Intelligent Surgery](https://mlsanity.com/paper/2506.02692/) (2025)
- [Navigating the Synthetic Realm: Harnessing Diffusion-based Models for Laparoscopic Text-to-Image Generation](https://mlsanity.com/paper/2312.03043/) (2023)
- [Why Deep Surgical Models Fail?: Revisiting Surgical Action Triplet Recognition through the Lens of Robustness](https://mlsanity.com/paper/2209.08647/) (2022)
- [Watch and Learn: Leveraging Expert Knowledge and Language for Surgical Video Understanding](https://mlsanity.com/paper/2503.11392/) (2025)
- [MEJO: MLLM-Engaged Surgical Triplet Recognition via Inter- and Intra-Task Joint Optimization](https://mlsanity.com/paper/2509.12893/) (2025)
- [Federated EndoViT: Pretraining Vision Transformers via Federated Learning on Endoscopic Image Collections](https://mlsanity.com/paper/2504.16612/) (2025)
- [fine-CLIP: Enhancing Zero-Shot Fine-Grained Surgical Action Recognition with Vision-Language Models](https://mlsanity.com/paper/2503.19670/) (2025)
- [DARIL: When Imitation Learning outperforms Reinforcement Learning in Surgical Action Planning](https://mlsanity.com/paper/2507.05011/) (2025)
- [Grounding Surgical Action Triplets with Instrument Instance Segmentation: A Dataset and Target-Aware Fusion Approach](https://mlsanity.com/paper/2511.00643/) (2025)
- [LapFM: A Laparoscopic Segmentation Foundation Model via Hierarchical Concept Evolving Pre-training](https://mlsanity.com/paper/2512.08439/) (2025)
- [TrajPred: Trajectory-Conditioned Joint Embedding Prediction for Surgical Instrument-Tissue Interaction Recognition in Vision-Language Models](https://mlsanity.com/paper/2603.06999/) (2026)
- [SurgWMBench: A Vision-Based Benchmark for World-Modeling Surgical Instrument Motion Planning](https://mlsanity.com/paper/2608.08070/) (2026)
- [CurConMix+: A Unified Spatio-Temporal Framework for Hierarchical Surgical Workflow Understanding](https://mlsanity.com/paper/2601.12312/) (2026)
- [Latent-Action-Guided Video-Language Feature Learning for Surgical Instrument-Tissue Interaction Recognition](https://mlsanity.com/paper/2607.19889/) (2026)
- [Optimize Surgical Triplet Recognition: A Knowledge-Driven Mixture-of-Experts Solution](https://mlsanity.com/paper/2608.22972/) (2026)
- [DART: Depth-as-Target Pretraining for Surgical Vision Foundation Models](https://mlsanity.com/paper/2609.04555/) (2026)
- [STSG-VQA: Evidence-Grounded Temporal Question Answering from Surgical Spatio-Temporal Scene Graphs](https://mlsanity.com/paper/2609.08543/) (2026)

## Abstract (arXiv)

In addition to generating data and annotations, devising sensible data splitting strategies and evaluation metrics is essential for the creation of a benchmark dataset. This practice ensures consensus on the usage of the data, homogeneous assessment, and uniform comparison of research methods on the dataset. This study focuses on CholecT50, which is a 50 video surgical dataset that formalizes surgical activities as triplets of <instrument, verb, target>. In this paper, we introduce the standard splits for the CholecT50 and CholecT45 datasets and show how they compare with existing use of the dataset. CholecT45 is the first public release of 45 videos of CholecT50 dataset. We also develop a metrics library, ivtmetrics, for model evaluation on surgical triplets. Furthermore, we conduct a benchmark study by reproducing baseline methods in the most predominantly used deep learning frameworks (PyTorch and TensorFlow) to evaluate them using the proposed data splits and metrics and release them publicly to support future research. The proposed data splits and evaluation metrics will enable global tracking of research progress on the dataset and facilitate optimal model selection for further deployment.