mlsanityData to 7 Oct 2026

Monocular pose estimation of articulated open surgery tools -- in the wild

arXiv 2407.12138 ↗First posted 16 Jul 2024cs.CV cs.LG cs.ROcited by 2

Not on the map. Why?

Abstract (arXiv)

This work presents a framework for monocular 6D pose estimation of surgical instruments in open surgery, addressing challenges such as object articulations, specularity, occlusions, and synthetic-to-real domain adaptation. The proposed approach consists of three main components: (1) synthetic data generation pipeline that incorporates 3D scanning of surgical tools with articulation rigging and physically-based rendering; (2) a tailored pose estimation framework combining tool detection with pose and articulation estimation; and (3) a training strategy on synthetic and real unannotated video data, employing domain adaptation with automatically generated pseudo-labels. Evaluations conducted on real data of open surgery demonstrate the good performance and real-world applicability of the proposed framework, highlighting its potential for integration into medical augmented reality and robotic systems. The approach eliminates the need for extensive manual annotation of real surgical data.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 34 of its 59 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

34 of 59 references.

References · 34

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2017Mask R-CNNcited by 7,259
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2016Pyramid Scene Parsing Networkcited by 2,472
  6. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  7. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  8. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  9. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 763
  10. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  11. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  12. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
22 more
  1. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
  2. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 302
  3. 2018Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomizationcited by 292
  4. 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 277
  5. 2021Recent Advances in Adversarial Training for Adversarial Robustnesscited by 276
  6. 2018BOP: Benchmark for 6D Object Pose Estimationcited by 254
  7. 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 237
  8. 2019Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimationcited by 218
  9. 2019DPOD: 6D Pose Object Detector and Refinercited by 189
  10. 20192017 Robotic Instrument Segmentation Challengecited by 166
  11. 2020EPOS: Estimating 6D Pose of Objects with Symmetriescited by 147
  12. 2019ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imagingcited by 145
  13. 2017Graph-Cut RANSACcited by 133
  14. 2022ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimationcited by 115
  15. 2020Gradient Centralization: A New Optimization Technique for Deep Neural Networkscited by 82
  16. 2023BOP Challenge 2022 on Detection, Segmentation and Pose Estimation of Specific Rigid Objectscited by 70
  17. 2023ContactGen: Generative Contact Modeling for Grasp Generationcited by 52
  18. 2019Progressive-X: Efficient, Anytime, Multi-Model Fitting Algorithmcited by 43
  19. 2020Synthetic and Real Inputs for Tool Segmentation in Robotic Surgerycited by 25
  20. 2022Using Hand Pose Estimation To Automate Open Surgery Training Feedbackcited by 6
  21. 2021MarkerPose: Robust Real-time Planar Target Tracking for Accurate Stereo Pose Estimationcited by 2
  22. 2015Convolutional Models for Joint Object Categorization and Pose Estimationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Estimating 2D Keypoints of Surgical Tools Using Vision-Language Models with Low-Rank Adaptationcited by 0
  2. 2026PICO: Projection-Informed Consistency Optimisation for 6DoF Surgical Tool Pose Estimationcited by 0