# Learning Invariant Representation of Tasks for Robust Surgical State Estimation

arXiv: https://arxiv.org/abs/2102.09119 · HTML: https://mlsanity.com/paper/2102.09119/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 18 Feb 2021.

## Builds on (most cited first, 18 of 18)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [Emergence of Invariance and Disentanglement in Deep Representations](https://mlsanity.com/paper/1706.01350/) (2017)
- [Temporal Convolutional Networks: A Unified Approach to Action Segmentation](https://mlsanity.com/paper/1608.08242/) (2016)
- [Controllable Invariance through Adversarial Feature Learning](https://mlsanity.com/paper/1705.11122/) (2017)
- [Disentangling factors of variation in deep representations using adversarial training](https://mlsanity.com/paper/1611.03383/) (2016)
- [Using 3D Convolutional Neural Networks to Learn Spatiotemporal Features for Automatic Surgical Gesture Recognition in Video](https://mlsanity.com/paper/1907.11454/) (2019)
- [Recognizing Surgical Activities with Recurrent Neural Networks](https://mlsanity.com/paper/1606.06329/) (2016)
- [Learning from a tiny dataset of manual annotations: a teacher/student approach for surgical phase recognition](https://mlsanity.com/paper/1812.00033/) (2018)
- [Unsupervised Adversarial Invariance](https://mlsanity.com/paper/1809.10083/) (2018)
- [Temporal Segmentation of Surgical Sub-tasks through Deep Learning with Multiple Data Sources](https://mlsanity.com/paper/2002.02921/) (2020)
- [daVinciNet: Joint Prediction of Motion and Surgical State in Robot-Assisted Surgery](https://mlsanity.com/paper/2009.11937/) (2020)
- [Symmetric Dilated Convolution for Surgical Gesture Recognition](https://mlsanity.com/paper/2007.06373/) (2020)
- [Unified Adversarial Invariance](https://mlsanity.com/paper/1905.03629/) (2019)
- [Robust speaker recognition using unsupervised adversarial invariance](https://mlsanity.com/paper/1911.00940/) (2019)
- [NIESR: Nuisance Invariant End-to-end Speech Recognition](https://mlsanity.com/paper/1907.03233/) (2019)

## Built on by (most cited first, 2 of 2)

- [TraSeTR: Track-to-Segment Transformer with Contrastive Query for Instance-level Instrument Segmentation in Robotic Surgery](https://mlsanity.com/paper/2202.08453/) (2022)
- [PEg TRAnsfer Workflow recognition challenge report: Does multi-modal data improve recognition?](https://mlsanity.com/paper/2202.05821/) (2022)

## Abstract (arXiv)

Surgical state estimators in robot-assisted surgery (RAS) - especially those trained via learning techniques - rely heavily on datasets that capture surgeon actions in laboratory or real-world surgical tasks. Real-world RAS datasets are costly to acquire, are obtained from multiple surgeons who may use different surgical strategies, and are recorded under uncontrolled conditions in highly complex environments. The combination of high diversity and limited data calls for new learning methods that are robust and invariant to operating conditions and surgical techniques. We propose StiseNet, a Surgical Task Invariance State Estimation Network with an invariance induction framework that minimizes the effects of variations in surgical technique and operating environments inherent to RAS datasets. StiseNet's adversarial architecture learns to separate nuisance factors from information needed for surgical state estimation. StiseNet is shown to outperform state-of-the-art state estimation methods on three datasets (including a new real-world RAS dataset: HERNIA-20).