# Deep Homography Estimation in Dynamic Surgical Scenes for Laparoscopic Camera Motion Extraction

arXiv: https://arxiv.org/abs/2109.15098 · HTML: https://mlsanity.com/paper/2109.15098/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 45 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 14 of 14)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [2017 Robotic Instrument Segmentation Challenge](https://mlsanity.com/paper/1902.06426/) (2019)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [2018 Robotic Scene Segmentation Challenge](https://mlsanity.com/paper/2001.11190/) (2020)
- [Unsupervised Deep Homography: A Fast and Robust Homography Estimation Model](https://mlsanity.com/paper/1709.03966/) (2017)
- [Content-Aware Unsupervised Deep Homography Estimation](https://mlsanity.com/paper/1909.05983/) (2019)
- [Deep Homography Estimation for Dynamic Scenes](https://mlsanity.com/paper/2004.02132/) (2020)
- [Heidelberg Colorectal Data Set for Surgical Data Science in the Sensor Operating Room](https://mlsanity.com/paper/2005.03501/) (2020)
- [Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery](https://mlsanity.com/paper/1805.02475/) (2018)
- [ToolNet: Holistically-Nested Real-Time Segmentation of Robotic Surgical Tools](https://mlsanity.com/paper/1706.08126/) (2017)
- [Image Compositing for Segmentation of Surgical Tools without Manual Annotations](https://mlsanity.com/paper/2102.09528/) (2021)
- [Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge](https://mlsanity.com/paper/2102.13644/) (2021)
- [ESAD: Endoscopic Surgeon Action Detection Dataset](https://mlsanity.com/paper/2006.07164/) (2020)

## Built on by (most cited first, 1 of 1)

- [Rapid and robust endoscopic content area estimation: A lean GPU-based pipeline and curated benchmark dataset](https://mlsanity.com/paper/2210.14771/) (2022)

## Abstract (arXiv)

Current laparoscopic camera motion automation relies on rule-based approaches or only focuses on surgical tools. Imitation Learning (IL) methods could alleviate these shortcomings, but have so far been applied to oversimplified setups. Instead of extracting actions from oversimplified setups, in this work we introduce a method that allows to extract a laparoscope holder's actions from videos of laparoscopic interventions. We synthetically add camera motion to a newly acquired dataset of camera motion free da Vinci surgery image sequences through a novel homography generation algorithm. The synthetic camera motion serves as a supervisory signal for camera motion estimation that is invariant to object and tool motion. We perform an extensive evaluation of state-of-the-art (SOTA) Deep Neural Networks (DNNs) across multiple compute regimes, finding our method transfers from our camera motion free da Vinci surgery dataset to videos of laparoscopic interventions, outperforming classical homography estimation approaches in both, precision by 41%, and runtime on a CPU by 43%.