# Visualizing Missing Surfaces In Colonoscopy Videos using Shared Latent Space Representations

arXiv: https://arxiv.org/abs/2101.07280 · HTML: https://mlsanity.com/paper/2101.07280/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 10 references that are in the map.
- First posted on arXiv 18 Jan 2021.

## Builds on (most cited first, 3 of 3)

- [Augmenting Colonoscopy using Extended and Directional CycleGAN for Lossy Image Translation](https://mlsanity.com/paper/2003.12473/) (2020)
- [Detecting Deficient Coverage in Colonoscopies](https://mlsanity.com/paper/2001.08589/) (2020)
- [Depth Reconstruction and Computer-Aided Polyp Detection in Optical Colonoscopy Video Frames](https://mlsanity.com/paper/1609.01329/) (2016)

## Built on by (most cited first, 2 of 2)

- [FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos](https://mlsanity.com/paper/2106.12522/) (2021)
- [Gaussian Process Prior Variational Autoencoder for Endoscopic Videos](https://mlsanity.com/paper/2606.19908/) (2026)

## Abstract (arXiv)

Optical colonoscopy (OC), the most prevalent colon cancer screening tool, has a high miss rate due to a number of factors, including the geometry of the colon (haustral fold and sharp bends occlusions), endoscopist inexperience or fatigue, endoscope field of view, etc. We present a framework to visualize the missed regions per-frame during the colonoscopy, and provides a workable clinical solution. Specifically, we make use of 3D reconstructed virtual colonoscopy (VC) data and the insight that VC and OC share the same underlying geometry but differ in color, texture and specular reflections, embedded in the OC domain. A lossy unpaired image-to-image translation model is introduced with enforced shared latent space for OC and VC. This shared latent space captures the geometric information while deferring the color, texture, and specular information creation to additional Gaussian noise input. This additional noise input can be utilized to generate one-to-many mappings from VC to OC and OC to OC. The code, data and trained models will be released via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.