# ProstaTD: Bridging Surgical Triplet from Classification to Fully Supervised Detection arXiv: https://arxiv.org/abs/2506.01130 ยท HTML: https://mlsanity.com/paper/2506.01130/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Builds on 28 of its 42 references that are in the map. - First posted on arXiv 1 Jun 2025. ## Builds on (most cited first, 28 of 28) - [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015) - [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020) - [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015) - [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017) - [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021) - [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022) - [DETRs Beat YOLOs on Real-time Object Detection](https://mlsanity.com/paper/2304.08069/) (2023) - [YOLOv10: Real-Time End-to-End Object Detection](https://mlsanity.com/paper/2405.14458/) (2024) - [YOLOv11: An Overview of the Key Architectural Enhancements](https://mlsanity.com/paper/2410.17725/) (2024) - [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016) - [YOLOv12: Attention-Centric Real-Time Object Detectors](https://mlsanity.com/paper/2502.12524/) (2025) - [Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videos](https://mlsanity.com/paper/2109.03223/) (2021) - [Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets](https://mlsanity.com/paper/2007.05405/) (2020) - [SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challenge](https://mlsanity.com/paper/2401.00496/) (2023) - [Pixel-Wise Recognition for Holistic Surgical Scene Understanding](https://mlsanity.com/paper/2401.11174/) (2024) - [CholecTriplet2021: A benchmark challenge for surgical action triplet recognition](https://mlsanity.com/paper/2204.04746/) (2022) - [The SARAS Endoscopic Surgeon Action Detection (ESAD) dataset: Challenges and methods](https://mlsanity.com/paper/2104.03178/) (2021) - [Latent Graph Representations for Critical View of Safety Assessment](https://mlsanity.com/paper/2212.04155/) (2022) - [MATIS: Masked-Attention Transformers for Surgical Instrument Segmentation](https://mlsanity.com/paper/2303.09514/) (2023) - [Towards Holistic Surgical Scene Understanding](https://mlsanity.com/paper/2212.04582/) (2022) - [MIcro-Surgical Anastomose Workflow recognition challenge report](https://mlsanity.com/paper/2103.13111/) (2021) - [Rendezvous in Time: An Attention-based Temporal Fusion approach for Surgical Triplet Recognition](https://mlsanity.com/paper/2211.16963/) (2022) - [CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection](https://mlsanity.com/paper/2302.06294/) (2023) - [Surgical Action Triplet Detection by Mixed Supervised Learning of Instrument-Tissue Interactions](https://mlsanity.com/paper/2307.09548/) (2023) - [CholecTrack20: A Multi-Perspective Tracking Dataset for Surgical Tools](https://mlsanity.com/paper/2312.07352/) (2023) - [Instrument-tissue Interaction Detection Framework for Surgical Video Understanding](https://mlsanity.com/paper/2404.00322/) (2024) - [MURPHY: Relations Matter in Surgical Workflow Analysis](https://mlsanity.com/paper/2212.12719/) (2022) - [SurgiTrack: Fine-Grained Multi-Class Multi-Tool Tracking in Surgical Videos](https://mlsanity.com/paper/2405.20333/) (2024) ## Built on by (most cited first, 4 of 4) - [SurgSigma: A Spectrum of Large-Scale Multimodal Data and Foundation Models for Surgical Intelligence](https://mlsanity.com/paper/2603.16822/) (2026) - [An Approach to Enriching Surgical Video Datasets for Fine-Grained Spatial-Temporal Understanding of Vision-Language Models](https://mlsanity.com/paper/2604.00784/) (2026) - [Latent-Action-Guided Video-Language Feature Learning for Surgical Instrument-Tissue Interaction Recognition](https://mlsanity.com/paper/2607.19889/) (2026) - [SPIRIT: Spatio-temporal Pairwise Relational Modeling of Instrument-Tissue Interactions for Surgical Action Triplet Recognition](https://mlsanity.com/paper/2608.02188/) (2026) ## Abstract (arXiv) Surgical triplet detection is a critical task in surgical video analysis. However, existing datasets like CholecT50 lack precise spatial bounding box annotations, rendering triplet classification at the image level insufficient for practical applications. The inclusion of bounding box annotations is essential to make this task meaningful, as they provide the spatial context necessary for accurate analysis and improved model generalizability. To address these shortcomings, we introduce ProstaTD, a large-scale, multi-institutional dataset for surgical triplet detection, developed from the technically demanding domain of robot-assisted prostatectomy. ProstaTD offers clinically defined temporal boundaries and high-precision bounding box annotations for each structured triplet activity. The dataset comprises 71,775 video frames and 196,490 annotated triplet instances, collected from 21 surgeries performed across multiple institutions, reflecting a broad range of surgical practices and intraoperative conditions. The annotation process was conducted under rigorous medical supervision and involved more than 60 contributors, including practicing surgeons and medically trained annotators, through multiple iterative phases of labeling and verification. To further facilitate future general-purpose surgical annotation, we developed two tailored labeling tools to improve efficiency and scalability in our annotation workflows. In addition, we created a surgical triplet detection evaluation toolkit that enables standardized and reproducible performance assessment across studies. ProstaTD is the largest and most diverse surgical triplet dataset to date, moving the field from simple classification to full detection with precise spatial and temporal boundaries and thereby providing a robust foundation for fair benchmarking.