Strategy-Supervised Autonomous Laparoscopic Camera Control via Event-Driven Graph Mining
Not on the map. Why?
Abstract (arXiv)
Autonomous laparoscopic camera control must maintain a stable and safe surgical view under rapid tool-tissue interactions while remaining interpretable to surgeons. We present a strategy-grounded framework that couples high-level vision-language inference with low-level closed-loop control. Offline, raw surgical videos are parsed into camera-relevant temporal events (e.g., interaction, working-distance deviation, and view-quality degradation) and structured as attributed event graphs. Mining these graphs yields a compact set of reusable camera-handling strategy primitives, which provide structured supervision for learning. Online, a fine-tuned Vision-Language Model (VLM) processes the live laparoscopic view to predict the dominant strategy and discrete image-based motion commands, executed by an IBVS-RCM controller under strict safety constraints; optional speech input enables intuitive human-in-the-loop conditioning. On a surgeon-annotated dataset, event parsing achieves reliable temporal localization (F1-score 0.86), and the mined strategies show strong semantic alignment with expert interpretation (cluster purity 0.81). Extensive ex vivo experiments on silicone phantoms and porcine tissues demonstrate that the proposed system outperforms junior surgeons in standardized camera-handling evaluations, reducing field-of-view centering error by 35.26% and image shaking by 62.33%, while preserving smooth motion and stable working-distance regulation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 55 references that are in the map. First posted on arXiv 24 Feb 2026.
Builds on
26 of 55 references.
References · 26
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2018An Algorithmic Perspective on Imitation Learningcited by 455
- 2016EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videoscited by 366
- 20192017 Robotic Instrument Segmentation Challengecited by 166
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoningcited by 144
- 2020TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networkscited by 139
- 2017Surgical Data Science: Enabling Next-Generation Surgerycited by 126
14 more
- 2021Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformercited by 93
- 2018Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learningcited by 81
- 2023LoViT: Long Video Transformer for Surgical Phase Recognitioncited by 56
- 2024Surgical-DINO: Adapter Learning of Foundation Models for Depth Estimation in Endoscopic Surgerycited by 52
- 2014Community Detection in Networks with Node Attributescited by 44
- 2022Towards Holistic Surgical Scene Understandingcited by 33
- 2024RoboNurse-VLA: Robotic Scrub Nurse System based on Vision-Language-Action Modelcited by 28
- 2025SRT-H: A Hierarchical Framework for Autonomous Surgery via Language Conditioned Imitation Learningcited by 23
- 2023CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detectioncited by 22
- 2024PitVis-2023 Challenge: Workflow Recognition in videos of Endoscopic Pituitary Surgerycited by 10
- 2022ARST: Auto-Regressive Surgical Transformer for Phase Recognition from Laparoscopic Videoscited by 6
- 2025LLM-Driven Augmented Reality Puppeteer: Controller-Free Voice-Commanded Robot Teleoperationcited by 4
- 2025SutureBot: A Precision Framework & Benchmark For Autonomous End-to-End Suturingcited by 4
- 2025AutoCam: Hierarchical Path Planning for an Autonomous Auxiliary Camera in Surgical Roboticscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence
- 2026SAW: Toward a Surgical Action World Model via Controllable and Scalable Video Generation
- 2026SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Video
- 2026SurgVLA-Bench: Towards Evaluating Vision-Language-Action Models for Laparoscopic Surgical Robotics
- 2025SurgXBench: Explainable Vision-Language Model Benchmark for Surgery
- 2026A vision-language model and platform for temporally mapping surgery from video
- 2024VISAGE: Video Synthesis using Action Graphs for Surgery
- 2021Deep Homography Estimation in Dynamic Surgical Scenes for Laparoscopic Camera Motion Extraction