mlsanityData to 7 Oct 2026

Know your sensORs -- A Modality Study For Surgical Action Classification

arXiv 2203.08674 ↗First posted 16 Mar 2022cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

The surgical operating room (OR) presents many opportunities for automation and optimization. Videos from various sources in the OR are becoming increasingly available. The medical community seeks to leverage this wealth of data to develop automated methods to advance interventional care, lower costs, and improve overall patient outcomes. Existing datasets from OR room cameras are thus far limited in size or modalities acquired, leaving it unclear which sensor modalities are best suited for tasks such as recognizing surgical action from videos. This study demonstrates that surgical action recognition performance can vary depending on the image modalities used. We perform a methodical analysis on several commonly available sensor modalities, presenting two fusion approaches that improve classification performance. The analyses are carried out on a set of multi-view RGB-D video recordings of 18 laparoscopic procedures.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 28 of its 74 references that are in the map. First posted on arXiv 16 Mar 2022.

Builds on

28 of 74 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  5. 2017The Kinetics Human Action Video Datasetcited by 1,894
  6. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  7. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  8. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  9. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 635
  10. 2016EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videoscited by 366
  11. 2018A Short Note about Kinetics-600cited by 331
  12. 2016The THUMOS Challenge on Action Recognition for Videos "in the Wild"cited by 213
16 more
  1. 2020TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networkscited by 139
  2. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 139
  3. 2020Surgical Data Science -- from Concepts toward Clinical Translationcited by 105
  4. 2019Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysiscited by 75
  5. 2020A Single Stream Network for Robust and Real-time RGB-D Salient Object Detectioncited by 67
  6. 2021OperA: Attention-Regularized Transformers for Surgical Phase Recognitioncited by 60
  7. 2018MVOR: A Multi-view RGB-D Operating Room Dataset for 2D and 3D Human Pose Estimationcited by 49
  8. 2019Using 3D Convolutional Neural Networks to Learn Spatiotemporal Features for Automatic Surgical Gesture Recognition in Videocited by 36
  9. 2020Automatic Operating Room Surgical Activity Recognition for Robot-Assisted Surgerycited by 34
  10. 2021PyTorchVideo: A Deep Learning Library for Video Understandingcited by 32
  11. 2020Project to Adapt: Domain Adaptation for Depth Completion from Noisy and Sparse Sensor Datacited by 28
  12. 2020A Robotic 3D Perception System for Operating Room Environment Awarenesscited by 13
  13. 2021Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in Challenging Environmentscited by 11
  14. 2019Surgical Gesture Recognition with Optical Flow onlycited by 7
  15. 2018Surgical Data Science: A Consensus Perspectivecited by 5
  16. 2021Efficient Global Optimization of Non-differentiable, Symmetric Objectives for Multi Camera Placementcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 3; 2026: 2.

6 papers.

  1. 2023DisguisOR: Holistic Face Anonymization for the Operating Roomcited by 9
  2. 2025Beyond Role-Based Surgical Domain Modeling: Generalizable Re-Identification in the Operating Roomcited by 6
  3. 2025TrackOR: Towards Personalized Intelligent Operating Rooms Through Robust Trackingcited by 2
  4. 2026COMPOSE: Hypergraph Cover Optimization for Multi-view 3D Human Pose Estimationcited by 1
  5. 2025Mitigating Biases in Surgical Operating Rooms with Geometrycited by 0
  6. 2026OR-Action: Multi-Role Video Understanding with Fine-Grained Actionscited by 0