# Unsupervised anomaly detection for a Smart Autonomous Robotic Assistant Surgeon (SARAS)using a deep residual autoencoder

arXiv: https://arxiv.org/abs/2104.11008 · HTML: https://mlsanity.com/paper/2104.11008/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 32 references that are in the map.
- First posted on arXiv 22 Apr 2021.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [Highway Networks](https://mlsanity.com/paper/1505.00387/) (2015)
- [Robust Unsupervised Video Anomaly Detection by Multi-Path Frame Prediction](https://mlsanity.com/paper/2011.02763/) (2020)
- [The SARAS Endoscopic Surgeon Action Detection (ESAD) dataset: Challenges and methods](https://mlsanity.com/paper/2104.03178/) (2021)
- [Unsupervised Video Anomaly Detection via Normalizing Flows with Implicit Latent Features](https://mlsanity.com/paper/2010.07524/) (2020)
- [ESAD: Endoscopic Surgeon Action Detection Dataset](https://mlsanity.com/paper/2006.07164/) (2020)

## Built on by (most cited first, 6 of 6)

- [Open challenges for Machine Learning based Early Decision-Making research](https://mlsanity.com/paper/2204.13111/) (2022)
- [SEDMamba: Enhancing Selective State Space Modelling with Bottleneck Mechanism and Fine-to-Coarse Temporal Fusion for Efficient Error Detection in Robot-Assisted Surgery](https://mlsanity.com/paper/2406.15920/) (2024)
- [Making the Flow Glow -- Robot Perception under Severe Lighting Conditions using Normalizing Flow Gradients](https://mlsanity.com/paper/2412.07565/) (2024)
- [Visual Anomaly Detection for Reliable Robotic Implantation of Flexible Microelectrode Array](https://mlsanity.com/paper/2510.09071/) (2025)
- [CataractCompDetect: Intraoperative Complication Detection in Cataract Surgery](https://mlsanity.com/paper/2511.18968/) (2025)
- [A Sensor-Adaptive Incremental Learning Framework for Artifact Detection in Satellite Precipitation Data](https://mlsanity.com/paper/2609.01514/) (2026)

## Abstract (arXiv)

Anomaly detection in Minimally-Invasive Surgery (MIS) traditionally requires a human expert monitoring the procedure from a console. Data scarcity, on the other hand, hinders what would be a desirable migration towards autonomous robotic-assisted surgical systems. Automated anomaly detection systems in this area typically rely on classical supervised learning. Anomalous events in a surgical setting, however, are rare, making it difficult to capture data to train a detection model in a supervised fashion. In this work we thus propose an unsupervised approach to anomaly detection for robotic-assisted surgery based on deep residual autoencoders. The idea is to make the autoencoder learn the 'normal' distribution of the data and detect abnormal events deviating from this distribution by measuring the reconstruction error. The model is trained and validated upon both the publicly available Cholec80 dataset, provided with extra annotation, and on a set of videos captured on procedures using artificial anatomies ('phantoms') produced as part of the Smart Autonomous Robotic Assistant Surgeon (SARAS) project. The system achieves recall and precision equal to 78.4%, 91.5%, respectively, on Cholec80 and of 95.6%, 88.1% on the SARAS phantom dataset. The end-to-end system was developed and deployed as part of the SARAS demonstration platform for real-time anomaly detection with a processing time of about 25 ms per frame.