# A deep learning framework for quality assessment and restoration in video endoscopy

arXiv: https://arxiv.org/abs/1904.07073 · HTML: https://mlsanity.com/paper/1904.07073/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2019.

## Built on by (most cited first, 24 of 24)

- [Robustness Analysis of Video-Language Models Against Visual and Language Perturbations](https://mlsanity.com/paper/2207.02159/) (2022)
- [SimCol3D -- 3D Reconstruction during Colonoscopy Challenge](https://mlsanity.com/paper/2307.11261/) (2023)
- [Panoptic Segmentation: A Review](https://mlsanity.com/paper/2111.10250/) (2021)
- [C3VDv2 -- Colonoscopy 3D video dataset with enhanced realism](https://mlsanity.com/paper/2506.24074/) (2025)
- [Artifact- and content-specific quality assessment for MRI with image rulers](https://mlsanity.com/paper/2111.03780/) (2021)
- [Multi-Scale Structural-aware Exposure Correction for Endoscopic Imaging](https://mlsanity.com/paper/2210.15033/) (2022)
- [How can we learn (more) from challenges? A statistical approach to driving future algorithm development](https://mlsanity.com/paper/2106.09302/) (2021)
- [An efficient semi-supervised quality control system trained using physics-based MRI-artefact generators and adversarial training](https://mlsanity.com/paper/2206.03359/) (2022)
- [AdaFNIO: Adaptive Fourier Neural Interpolation Operator for video frame interpolation](https://mlsanity.com/paper/2211.10791/) (2022)
- [Depth-Aware Endoscopic Video Inpainting](https://mlsanity.com/paper/2407.02675/) (2024)
- [Surgical Scene Segmentation by Transformer With Asymmetric Feature Enhancement](https://mlsanity.com/paper/2410.17642/) (2024)
- [Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipation](https://mlsanity.com/paper/2412.06454/) (2024)
- [Uncertainty-Aware Regularization for Image-to-Image Translation](https://mlsanity.com/paper/2412.01705/) (2024)
- [SST-EM: Advanced Metrics for Evaluating Semantic, Spatial and Temporal Aspects in Video Editing](https://mlsanity.com/paper/2501.07554/) (2025)
- [Temporal Propagation of Asymmetric Feature Pyramid for Surgical Scene Segmentation](https://mlsanity.com/paper/2504.13440/) (2025)
- [Colon-X: Advancing Intelligent Colonoscopy toward Clinical Reasoning](https://mlsanity.com/paper/2512.03667/) (2025)
- [Colorectal Polyp Detection in Real-world Scenario: Design and Experiment Study](https://mlsanity.com/paper/2101.04034/) (2021)
- [Specular reflections removal in colposcopic images based on neural networks: Supervised training with no ground truth previous knowledge](https://mlsanity.com/paper/2106.02221/) (2021)
- [EndoBoost: a plug-and-play module for false positive suppression during computer-aided polyp detection in real-world colonoscopy (with dataset)](https://mlsanity.com/paper/2212.12204/) (2022)
- [Focus-to-Perceive Representation Learning: A Cognition-Inspired Hierarchical Framework for Endoscopic Video Analysis](https://mlsanity.com/paper/2603.25778/) (2026)
- [HD-DinoMoE: A Class-Aware Hierarchical Dual Mixture-of-Experts Network for Scleral Anomaly Segmentation in Complex Acquisition Scenarios](https://mlsanity.com/paper/2606.04888/) (2026)
- [C3VD-DEFCOL: A Deformable Colonoscopy Dataset with Time-Resolved 3D Ground Truth and Realistic Appearance](https://mlsanity.com/paper/2606.07891/) (2026)
- [Expert Consensus on Criteria for the Automated Assessment of Laparoscopic Camera Navigation](https://mlsanity.com/paper/2606.23131/) (2026)
- [On the Robustness of Temporal Vision-Language Models for Surgical Endoscopy Videos](https://mlsanity.com/paper/2608.14262/) (2026)

## Abstract (arXiv)

Endoscopy is a routine imaging technique used for both diagnosis and minimally invasive surgical treatment. Artifacts such as motion blur, bubbles, specular reflections, floating objects and pixel saturation impede the visual interpretation and the automated analysis of endoscopy videos. Given the widespread use of endoscopy in different clinical applications, we contend that the robust and reliable identification of such artifacts and the automated restoration of corrupted video frames is a fundamental medical imaging problem. Existing state-of-the-art methods only deal with the detection and restoration of selected artifacts. However, typically endoscopy videos contain numerous artifacts which motivates to establish a comprehensive solution. We propose a fully automatic framework that can: 1) detect and classify six different primary artifacts, 2) provide a quality score for each frame and 3) restore mildly corrupted frames. To detect different artifacts our framework exploits fast multi-scale, single stage convolutional neural network detector. We introduce a quality metric to assess frame quality and predict image restoration success. Generative adversarial networks with carefully chosen regularization are finally used to restore corrupted frames. Our detector yields the highest mean average precision (mAP at 5% threshold) of 49.0 and the lowest computational time of 88 ms allowing for accurate real-time processing. Our restoration models for blind deblurring, saturation correction and inpainting demonstrate significant improvements over previous methods. On a set of 10 test videos we show that our approach preserves an average of 68.7% which is 25% more frames than that retained from the raw videos.