mlsanityData to 7 Oct 2026

EgoSurgery-Tool: A Dataset of Surgical Tool and Hand Detection from Egocentric Open Surgery Videos

arXiv 2406.03095 ↗First posted 5 Jun 2024cs.CV cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Surgical tool detection is a fundamental task for understanding egocentric open surgery videos. However, detecting surgical tools presents significant challenges due to their highly imbalanced class distribution, similar shapes and similar textures, and heavy occlusion. The lack of a comprehensive large-scale dataset compounds these challenges. In this paper, we introduce EgoSurgery-Tool, an extension of the existing EgoSurgery-Phase dataset, which contains real open surgery videos captured using an egocentric camera attached to the surgeon's head, along with phase annotations. EgoSurgery-Tool has been densely annotated with surgical tools and comprises over 49K surgical tool bounding boxes across 15 categories, constituting a large-scale surgical tool detection dataset. EgoSurgery-Tool also provides annotations for hand detection with over 46K hand-bounding boxes, capturing hand-object interactions that are crucial for understanding activities in egocentric open surgery. EgoSurgery-Tool is superior to existing datasets due to its larger scale, greater variety of surgical tools, more annotations, and denser scenes. We conduct a comprehensive analysis of EgoSurgery-Tool using nine popular object detectors to assess their effectiveness in both surgical tool and hand detection. The dataset will be released at https://github.com/Fujiry0/EgoSurgery.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 18 of its 25 references that are in the map. First posted on arXiv 5 Jun 2024.

Builds on

18 of 25 references.

References · 18

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  4. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  5. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  6. 2019Objects as Pointscited by 1,172
  7. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 494
  8. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  9. 2020VarifocalNet: An IoU-aware Dense Object Detectorcited by 230
  10. 2023Dense Distinct Query for End-to-End Object Detectioncited by 76
  11. 2018Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networkscited by 56
  12. 2020Detection and Localization of Robotic Tools in Robot-Assisted Surgery Videos Using Deep Neural Networks for Region Proposal and Detectioncited by 22
6 more
  1. 2018Weakly-Supervised Learning for Tool Localization in Laparoscopic Videoscited by 20
  2. 2024EgoSurgery-Phase: A Dataset of Surgical Phase Recognition from Egocentric Open Surgery Videoscited by 20
  3. 2023Intuitive Surgical SurgToolLoc and SurgVU Challenges Results: 2022-2025cited by 11
  4. 2020Using Computer Vision to Automate Hand Detection and Tracking of Surgeon Movements in Videos of Open Surgerycited by 9
  5. 2022A semi-supervised Teacher-Student framework for surgical tool detection and localizationcited by 6
  6. 2024Weakly Semi-supervised Tool Detection in Minimally Invasive Surgery Videoscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025EgoLife: Towards Egocentric Life Assistantcited by 86
  2. 2025EgoExOR: An Ego-Exo-Centric Operating Room Dataset for Surgical Activity Understandingcited by 11
  3. 2025Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Reviewcited by 8
  4. 2025EgoSurgery-HTS: A Dataset for Egocentric Hand-Tool Segmentation in Open Surgery Videoscited by 3
  5. 2026YOLOv10-Based Multi-Task Framework for Hand Localization and Laterality Classification in Surgical Videoscited by 0
  6. 2026AnyGroundBench: A Specialized-Domain Benchmark for Video Grounding in Vision-Language Modelscited by 0
  7. 2026RefineRank: Joint Box Refinement and Ranking for Surgical Spatio-Temporal Groundingcited by 0