mlsanityData to 14 Oct 2026

Self-Supervised Surgical Instrument 3D Reconstruction from a Single Camera Image

arXiv 2211.14467 ↗First posted 26 Nov 2022cs.CV eess.IVcited by 6

Not on the map. Why?

Abstract (arXiv)

Surgical instrument tracking is an active research area that can provide surgeons feedback about the location of their tools relative to anatomy. Recent tracking methods are mainly divided into two parts: segmentation and object detection. However, both can only predict 2D information, which is limiting for application to real-world surgery. An accurate 3D surgical instrument model is a prerequisite for precise predictions of the pose and depth of the instrument. Recent single-view 3D reconstruction methods are only used in natural object reconstruction and do not achieve satisfying reconstruction accuracy without 3D attribute-level supervision. Further, those methods are not suitable for the surgical instruments because of their elongated shapes. In this paper, we firstly propose an end-to-end surgical instrument reconstruction system -- Self-supervised Surgical Instrument Reconstruction (SSIR). With SSIR, we propose a multi-cycle-consistency strategy to help capture the texture information from a slim instrument while only requiring a binary instrument label map. Experiments demonstrate that our approach improves the reconstruction quality of surgical instruments compared to other self-supervised methods and achieves promising results.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 26 Nov 2022.

Builds on

14 of 24 references.

References · 14

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Spatial Transformer Networkscited by 1,429
  4. 2018Object Detection with Deep Learning: A Reviewcited by 528
  5. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  6. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  7. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  8. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  9. 2019Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Eracited by 87
  10. 2020DC-UNet: Rethinking the U-Net Architecture with Dual Channel Efficient CNN for Medical Images Segmentationcited by 28
  11. 2022Min-Max Similarity: A Contrastive Semi-Supervised Deep Learning Network for Surgical Tools Segmentationcited by 28
  12. 2021CFPNet: Channel-wise Feature Pyramid for Real-Time Semantic Segmentationcited by 14
2 more
  1. 2021CFPNet-M: A Light-Weight Encoder-Decoder Based Network for Multimodal Biomedical Image Real-Time Segmentationcited by 11
  2. 2020Segmentation of Infrared Breast Images Using MultiResUnet Neural Networkcited by 4

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 4.

6 papers.

  1. 2024Zero-Shot Surgical Tool Segmentation in Monocular Video Using Segment Anything Model 2cited by 11
  2. 2024NeuroBOLT: Resting-state EEG-to-fMRI Synthesis with Multi-dimensional Feature Mappingcited by 11
  3. 2023WS-SfMLearner: Self-supervised Monocular Depth and Ego-motion Estimation on Surgical Videos with Unknown Camera Parameterscited by 9
  4. 2023SAMSNeRF: Segment Anything Model (SAM) Guides Dynamic Surgical Scene Reconstruction by Neural Radiance Field (NeRF)cited by 8
  5. 2024Surgical Depth Anything: Depth Estimation for Surgical Scenes using Foundation Modelscited by 5
  6. 2024DaRePlane: Direction-aware Representations for Dynamic Scene Reconstructioncited by 0