# FoldIt: Haustral Folds Detection and Segmentation in Colonoscopy Videos

arXiv: https://arxiv.org/abs/2106.12522 · HTML: https://mlsanity.com/paper/2106.12522/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 20 references that are in the map.
- First posted on arXiv 23 Jun 2021.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training](https://mlsanity.com/paper/1711.06606/) (2017)
- [TraVeLGAN: Image-to-image Translation by Transformation Vector Learning](https://mlsanity.com/paper/1902.09631/) (2019)
- [Augmenting Colonoscopy using Extended and Directional CycleGAN for Lossy Image Translation](https://mlsanity.com/paper/2003.12473/) (2020)
- [SLAM Endoscopy enhanced by adversarial depth prediction](https://mlsanity.com/paper/1907.00283/) (2019)
- [Reconstructing Sinus Anatomy from Endoscopic Video -- Towards a Radiation-free Approach for Quantitative Longitudinal Assessment](https://mlsanity.com/paper/2003.08502/) (2020)
- [VR-Caps: A Virtual Environment for Capsule Endoscopy](https://mlsanity.com/paper/2008.12949/) (2020)
- [Detecting Deficient Coverage in Colonoscopies](https://mlsanity.com/paper/2001.08589/) (2020)
- [Depth Reconstruction and Computer-Aided Polyp Detection in Optical Colonoscopy Video Frames](https://mlsanity.com/paper/1609.01329/) (2016)
- [Corresponding Supine and Prone Colon Visualization Using Eigenfunction Analysis and Fold Modeling](https://mlsanity.com/paper/1810.08850/) (2018)
- [Visualizing Missing Surfaces In Colonoscopy Videos using Shared Latent Space Representations](https://mlsanity.com/paper/2101.07280/) (2021)

## Built on by (most cited first, 5 of 5)

- [SegCol Challenge: Semantic Segmentation for Tools and Fold Edges in Colonoscopy data](https://mlsanity.com/paper/2412.16078/) (2024)
- [CLTS-GAN: Color-Lighting-Texture-Specular Reflection Augmentation for Colonoscopy](https://mlsanity.com/paper/2206.14951/) (2022)
- [ColonCrafter: A Depth Estimation Model for Colonoscopy Videos Using Diffusion Priors](https://mlsanity.com/paper/2509.13525/) (2025)
- [Sim2Real in endoscopy segmentation with a novel structure aware image translation](https://mlsanity.com/paper/2505.02654/) (2025)
- [RT-GAN: Recurrent Temporal GAN for Adding Lightweight Temporal Consistency to Frame-Based Domain Translation Approaches](https://mlsanity.com/paper/2310.00868/) (2023)

## Abstract (arXiv)

Haustral folds are colon wall protrusions implicated for high polyp miss rate during optical colonoscopy procedures. If segmented accurately, haustral folds can allow for better estimation of missed surface and can also serve as valuable landmarks for registering pre-treatment virtual (CT) and optical colonoscopies, to guide navigation towards the anomalies found in pre-treatment scans. We present a novel generative adversarial network, FoldIt, for feature-consistent image translation of optical colonoscopy videos to virtual colonoscopy renderings with haustral fold overlays. A new transitive loss is introduced in order to leverage ground truth information between haustral fold annotations and virtual colonoscopy renderings. We demonstrate the effectiveness of our model on real challenging optical colonoscopy videos as well as on textured virtual colonoscopy videos with clinician-verified haustral fold annotations. All code and scripts to reproduce the experiments of this paper will be made available via our Computational Endoscopy Platform at https://github.com/nadeemlab/CEP.