Surgical Data Science -- from Concepts toward Clinical Translation
Not on the map. Why?
Abstract (arXiv)
Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.
Key facts. Cited by 105 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Oct 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 2; 2022: 14; 2023: 21; 2024: 27; 2025: 24; 2026: 17.
105 papers.
Sort:
By area:
- 2022AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomycited by 85
- 2023Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lecturescited by 68
- 2021Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmarkcited by 58
- 2022Dissecting Self-Supervised Learning Methods for Surgical Computer Visioncited by 48
- 2024Pixel-Wise Recognition for Holistic Surgical Scene Understandingcited by 48
- 2024Surgical SAM 2: Real-time Segment Anything in Surgical Video by Efficient Frame Pruningcited by 46
- 2023Advancing Surgical VQA with Scene Graph Knowledgecited by 41
- 2023Challenges in Multi-centric Generalization: Phase and Step Recognition in Roux-en-Y Gastric Bypass Surgerycited by 40
- 2024Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentationcited by 38
- 2023Tracking and Mapping in Medical Computer Vision: A Reviewcited by 36
- 2022Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentationcited by 27
- 2025Scaling up self-supervised learning for improved surgical foundation modelscited by 25
38 more
- 2023The Endoscapes Dataset for Surgical Scene Segmentation, Object Detection, and Critical View of Safety Assessment: Official Splits and Benchmarkcited by 21
- 2022Labeling instructions matter in biomedical image analysiscited by 20
- 2022On the Pitfalls of Batch Normalization for End-to-End Video Learning: A Study on Surgical Workflow Analysiscited by 19
- 2025LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settingscited by 19
- 2022Federated Cycling (FedCy): Semi-supervised Federated Learning of Surgical Phasescited by 18
- 2023Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the artcited by 18
- 2023SimCol3D -- 3D Reconstruction during Colonoscopy Challengecited by 18
- 2023Human-in-the-loop Embodied Intelligence with Interactive Simulation Environment for Surgical Robot Learningcited by 11
- 2024PitVis-2023 Challenge: Workflow Recognition in videos of Endoscopic Pituitary Surgerycited by 10
- 2025Large-scale Self-supervised Video Foundation Model for Intelligent Surgerycited by 10
- 2023Self-distillation for surgical action recognitioncited by 9
- 2023Exploring Semantic Consistency in Unpaired Image Translation to Generate Data for Surgical Applicationscited by 9
- 2024Creating a Digital Twin of Spinal Surgery: A Proof of Conceptcited by 9
- 2024Online 3D reconstruction and dense tracking in endoscopic videoscited by 9
- 2024VISAGE: Video Synthesis using Action Graphs for Surgerycited by 9
- 2023Jumpstarting Surgical Computer Visioncited by 8
- 2024MuST: Multi-Scale Transformers for Surgical Phase Recognitioncited by 8
- 2024Data Augmentation for Surgical Scene Segmentation with Anatomy-Aware Diffusion Modelscited by 8
- 2025Video Dataset for Surgical Phase, Keypoint, and Instrument Recognition in Laparoscopic Surgery (PhaKIR)cited by 8
- 20223D Human Pose Estimation in Multi-View Operating Room Videos Using Differentiable Camera Projectionscited by 7
- 2024OSSAR: Towards Open-Set Surgical Activity Recognition in Robot-assisted Surgerycited by 7
- 2024VS-Assistant: Versatile Surgery Assistant on the Demand of Surgeonscited by 7
- 2024SegSTRONG-C: Segmenting Surgical Tools Robustly On Non-adversarial Generated Corruptions -- An EndoVis'24 Challengecited by 7
- 2024Exploring the Effect of Dataset Diversity in Self-Supervised Learning for Surgical Computer Visioncited by 7
- 2022Know your sensORs -- A Modality Study For Surgical Action Classificationcited by 6
- 2023TUNeS: A Temporal U-Net with Self-Attention for Video-based Surgical Phase Recognitioncited by 6
- 2023Surgical Temporal Action-aware Network with Sequence Regularization for Phase Recognitioncited by 6
- 2023ST(OR)2: Spatio-Temporal Object Level Reasoning for Activity Recognition in the Operating Roomcited by 6
- 2025Beyond Role-Based Surgical Domain Modeling: Generalizable Re-Identification in the Operating Roomcited by 6
- 2025The SAGES Critical View of Safety Challenge: A Global Benchmark for AI-Assisted Surgical Quality Assessmentcited by 6
- 2022Bounded Future MS-TCN++ for surgical gesture recognitioncited by 5
- 2024Multitask Learning in Minimally Invasive Surgical Vision: A Reviewcited by 5
- 2024SR-Mamba: Effective Surgical Phase Recognition with State Space Modelcited by 5
- 2023VideoSum: A Python Library for Surgical Video Summarizationcited by 4
- 2023A Client-server Deep Federated Learning for Cross-domain Surgical Image Segmentationcited by 4
- 2024Surgical Triplet Recognition via Diffusion Modelcited by 4
- 2024Think Step by Step: Chain-of-Gesture Prompting for Error Detection in Robotic Surgical Videoscited by 4
- 2024SurgicaL-CD: Generating Surgical Images via Unpaired Image Translation with Latent Consistency Diffusion Modelscited by 4
50 most cited of 105 All 105 (JSON)
- 2026A Surgical Foundation Model Reveals Task-Dependent Label Efficiency25 Sep 2026
- 2026Structured Reasoning Agentic Framework for Interpretable Critical View of Safety Assessment25 Sep 2026
- 2026MedVLA: A Hierarchical Vision-Language-Action Framework for Closed-Loop Precision Medical Robot Manipulation22 Sep 2026
- 2026OphBiWSSD: Scaling Temporal Action Localization in Ophthalmic Surgeries with Bidirectional Weight-tied State Space Duality11 Sep 2026
- 2026Surgical Video Generation From Diffusion to World Models: A Survey26 Aug 2026
- 2026Bootstrapping Vision-Language Model for Hysteroscopic Surgical Scene Segmentation10 Aug 2026
- 2026Extended KAFR: A kinematic-adaptive paradigm for the efficient analysis of surgical video2 Aug 2026
- 2026HyperVLP: Enhancing Hierarchical Surgical Video-Language Pre-training in Hyperbolic Space30 Jun 2026
- 2026Surgical Anatomy Recognition with Context Learning using Foundation Representations20 Jun 2026
- 2026AI for Quality Assurance in the Operating Room16 Jun 2026
- 2026Spectral Vision Transformer for Efficient Tokenization with Limited Data12 May 2026
- 2026SurgCoT: Advancing Spatiotemporal Reasoning in Surgical Videos through a Chain-of-Thought Benchmark22 Apr 2026
38 more
- 2026AffordTissue: Dense Affordance Prediction for Tool-Action Specific Tissue Interaction1 Apr 2026
- 2026CliPPER: Contextual Video-Language Pretraining on Long-form Intraoperative Surgical Procedures for Event Recognition25 Mar 2026
- 2026Surg-R1: A Hierarchical Reasoning Foundation Model for Scalable and Interpretable Surgical Decision Support with Multi-Center Clinical Validation12 Mar 2026
- 2026Attachment Anchors: A Novel Framework for Laparoscopic Grasping Point Prediction in Colorectal Surgery19 Feb 2026
- 2026Self-Supervised Uncalibrated Multi-View Video Anonymization in the Operating Room2 Feb 2026
- 2025Toward Real-Time Surgical Scene Segmentation via a Spike-Driven Video Transformer with Spike-Informed Pretraining24 Dec 2025
- 2025SurgSLOT: Segment Anything in Surgical Videos via Semantic Long-term Tracking20 Nov 2025
- 2025End to End AI System for Surgical Gesture Sequence Recognition and Clinical Outcome Prediction14 Nov 2025
- 2025Video Dataset for Surgical Phase, Keypoint, and Instrument Recognition in Laparoscopic Surgery (PhaKIR)9 Nov 2025
- 2025SurgiATM: A Physics-Guided Plug-and-Play Model for Deep Learning-Based Smoke Removal in Laparoscopic Surgery7 Nov 2025
- 2025Federated Learning for Surgical Vision in Appendicitis Classification: Results of the FedSurg EndoVis 2024 Challenge6 Oct 2025
- 2025The SAGES Critical View of Safety Challenge: A Global Benchmark for AI-Assisted Surgical Quality Assessment21 Sep 2025
- 2025Exploring Pre-training Across Domains for Few-Shot Surgical Skill Assessment11 Sep 2025
- 2025TrackOR: Towards Personalized Intelligent Operating Rooms Through Robust Tracking11 Aug 2025
- 2025Endoscopic Depth Estimation Based on Deep Learning: A Survey28 Jul 2025
- 2025Learning from Sparse Point Labels for Dense Carcinosis Localization in Advanced Ovarian Cancer Assessment9 Jul 2025
- 2025Surgeons Awareness, Expectations, and Involvement with Artificial Intelligence: a Survey Pre and Post the GPT Era9 Jun 2025
- 2025Learning dissection trajectories from expert surgical videos via imitation learning with equivariant diffusion5 Jun 2025
- 2025Large-scale Self-supervised Video Foundation Model for Intelligent Surgery3 Jun 2025
- 2025SemiVT-Surge: Semi-Supervised Video Transformer for Surgical Phase Recognition2 Jun 2025
- 2025Mission Balance: Generating Under-represented Class Samples using Video Diffusion Models14 May 2025
- 2025Federated EndoViT: Pretraining Vision Transformers via Federated Learning on Endoscopic Image Collections23 Apr 2025
- 2025LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settings25 Mar 2025
- 2025Beyond Role-Based Surgical Domain Modeling: Generalizable Re-Identification in the Operating Room17 Mar 2025
- 2025CoStoDet-DDPM: Collaborative Training of Stochastic and Deterministic Models Improves Surgical Workflow Anticipation and Recognition13 Mar 2025
- 2025Multi-view Video-Pose Pretraining for Operating Room Surgical Activity Recognition19 Feb 2025
- 2025Early Operative Difficulty Assessment in Laparoscopic Cholecystectomy via Snapshot-Centric Video Analysis10 Feb 2025
- 2025Text-driven Adaptation of Foundation Models for Few-shot Surgical Workflow Analysis16 Jan 2025
- 2025Scaling up self-supervised learning for improved surgical foundation models16 Jan 2025
- 2024Benchmarking and Enhancing Surgical Phase Recognition Models for Robotic-Assisted Esophagectomy5 Dec 2024
- 2024AMNCutter: Affinity-Attention-Guided Multi-View Normalized Cutter for Unsupervised Surgical Instrument Segmentation6 Nov 2024
- 2024Towards Robust Algorithms for Surgical Phase Recognition via Digital Twin Representation26 Oct 2024
- 2024VISAGE: Video Synthesis using Action Graphs for Surgery23 Oct 2024
- 2024Data Augmentation for Surgical Scene Segmentation with Anatomy-Aware Diffusion Models10 Oct 2024
- 2024Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation30 Sep 2024
- 2024Leveraging Pre-trained Models for Robust Federated Learning for Kidney Stone Type Recognition30 Sep 2024
- 2024SurgPLAN++: Universal Surgical Phase Localization Network for Online and Offline Inference19 Sep 2024
- 2024Online 3D reconstruction and dense tracking in endoscopic videos9 Sep 2024
50 newest of 105 All 105 (JSON)
- 2024Data Augmentation for Surgical Scene Segmentation with Anatomy-Aware Diffusion Modelscited by 8
- 2024Surgical Triplet Recognition via Diffusion Modelcited by 4
- 2024SurgicaL-CD: Generating Surgical Images via Unpaired Image Translation with Latent Consistency Diffusion Modelscited by 4
- 2025Mission Balance: Generating Under-represented Class Samples using Video Diffusion Modelscited by 1
- 2025Learning dissection trajectories from expert surgical videos via imitation learning with equivariant diffusioncited by 1
- 2025CoStoDet-DDPM: Collaborative Training of Stochastic and Deterministic Models Improves Surgical Workflow Anticipation and Recognitioncited by 0
- 2025Learning dissection trajectories from expert surgical videos via imitation learning with equivariant diffusion5 Jun 2025
- 2025Mission Balance: Generating Under-represented Class Samples using Video Diffusion Models14 May 2025
- 2025CoStoDet-DDPM: Collaborative Training of Stochastic and Deterministic Models Improves Surgical Workflow Anticipation and Recognition13 Mar 2025
- 2024Data Augmentation for Surgical Scene Segmentation with Anatomy-Aware Diffusion Models10 Oct 2024
- 2024SurgicaL-CD: Generating Surgical Images via Unpaired Image Translation with Latent Consistency Diffusion Models19 Aug 2024
- 2024Surgical Triplet Recognition via Diffusion Model19 Jun 2024
- 2024Think Step by Step: Chain-of-Gesture Prompting for Error Detection in Robotic Surgical Videoscited by 4
- 2026Surg-R1: A Hierarchical Reasoning Foundation Model for Scalable and Interpretable Surgical Decision Support with Multi-Center Clinical Validationcited by 2
- 2026SurgCoT: Advancing Spatiotemporal Reasoning in Surgical Videos through a Chain-of-Thought Benchmarkcited by 0
- 2026Structured Reasoning Agentic Framework for Interpretable Critical View of Safety Assessmentcited by 0
- 2026Structured Reasoning Agentic Framework for Interpretable Critical View of Safety Assessment25 Sep 2026
- 2026SurgCoT: Advancing Spatiotemporal Reasoning in Surgical Videos through a Chain-of-Thought Benchmark22 Apr 2026
- 2026Surg-R1: A Hierarchical Reasoning Foundation Model for Scalable and Interpretable Surgical Decision Support with Multi-Center Clinical Validation12 Mar 2026
- 2024Think Step by Step: Chain-of-Gesture Prompting for Error Detection in Robotic Surgical Videos27 Jun 2024
- 2022AutoLaparo: A New Dataset of Integrated Multi-tasks for Image-guided Surgical Automation in Laparoscopic Hysterectomycited by 85
- 2023Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lecturescited by 68
- 2021Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmarkcited by 58
- 2022Dissecting Self-Supervised Learning Methods for Surgical Computer Visioncited by 48
- 2024Pixel-Wise Recognition for Holistic Surgical Scene Understandingcited by 48
- 2024Surgical SAM 2: Real-time Segment Anything in Surgical Video by Efficient Frame Pruningcited by 46
- 2023Advancing Surgical VQA with Scene Graph Knowledgecited by 41
- 2023Challenges in Multi-centric Generalization: Phase and Step Recognition in Roux-en-Y Gastric Bypass Surgerycited by 40
- 2024Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentationcited by 38
- 2023Tracking and Mapping in Medical Computer Vision: A Reviewcited by 36
- 2022Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentationcited by 27
- 2025Scaling up self-supervised learning for improved surgical foundation modelscited by 25
12 most cited of 90 All 90 (JSON)
- 2026A Surgical Foundation Model Reveals Task-Dependent Label Efficiency25 Sep 2026
- 2026OphBiWSSD: Scaling Temporal Action Localization in Ophthalmic Surgeries with Bidirectional Weight-tied State Space Duality11 Sep 2026
- 2026Extended KAFR: A kinematic-adaptive paradigm for the efficient analysis of surgical video2 Aug 2026
- 2026Surgical Anatomy Recognition with Context Learning using Foundation Representations20 Jun 2026
- 2026AI for Quality Assurance in the Operating Room16 Jun 2026
- 2026Spectral Vision Transformer for Efficient Tokenization with Limited Data12 May 2026
- 2026AffordTissue: Dense Affordance Prediction for Tool-Action Specific Tissue Interaction1 Apr 2026
- 2026CliPPER: Contextual Video-Language Pretraining on Long-form Intraoperative Surgical Procedures for Event Recognition25 Mar 2026
- 2026Attachment Anchors: A Novel Framework for Laparoscopic Grasping Point Prediction in Colorectal Surgery19 Feb 2026
- 2026Self-Supervised Uncalibrated Multi-View Video Anonymization in the Operating Room2 Feb 2026
- 2025Toward Real-Time Surgical Scene Segmentation via a Spike-Driven Video Transformer with Spike-Informed Pretraining24 Dec 2025
- 2025SurgSLOT: Segment Anything in Surgical Videos via Semantic Long-term Tracking20 Nov 2025
12 newest of 90 All 90 (JSON)