# Automatic Estimation of Ulcerative Colitis Severity from Endoscopy Videos using Ordinal Multi-Instance Learning

arXiv: https://arxiv.org/abs/2109.14685 · HTML: https://mlsanity.com/paper/2109.14685/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 18 references that are in the map.
- First posted on arXiv 29 Sep 2021.

## Builds on (most cited first, 4 of 4)

- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [Localization of Critical Findings in Chest X-Ray without Local Annotations Using Multi-Instance Learning](https://mlsanity.com/paper/2001.08817/) (2020)
- [PS-DeVCEM: Pathology-sensitive deep learning model for video capsule endoscopy based on weakly labeled data](https://mlsanity.com/paper/2011.12957/) (2020)
- [Extracting 2D weak labels from volume labels using multiple instance learning in CT hemorrhage detection](https://mlsanity.com/paper/1911.05650/) (2019)

## Built on by (most cited first, 7 of 7)

- [Class Distance Weighted Cross-Entropy Loss for Ulcerative Colitis Severity Estimation](https://mlsanity.com/paper/2202.05167/) (2022)
- [Ordinal Multiple-instance Learning for Ulcerative Colitis Severity Estimation with Selective Aggregated Transformer](https://mlsanity.com/paper/2411.14750/) (2024)
- [Deep Bayesian Active-Learning-to-Rank for Endoscopic Image Data](https://mlsanity.com/paper/2208.03020/) (2022)
- [Learning from Majority Label: A Novel Problem in Multi-class Multiple-Instance Learning](https://mlsanity.com/paper/2509.04023/) (2025)
- [Arges: Spatio-Temporal Transformer for Ulcerative Colitis Severity Assessment in Endoscopy Videos](https://mlsanity.com/paper/2410.00536/) (2024)
- [MEGAN: Mixture of Experts for Robust Uncertainty Estimation in Endoscopy Videos](https://mlsanity.com/paper/2509.12772/) (2025)
- [Domain Adaptation for Ulcerative Colitis Severity Estimation Using Patient-Level Diagnoses](https://mlsanity.com/paper/2509.14573/) (2025)

## Abstract (arXiv)

Ulcerative colitis (UC) is a chronic inflammatory bowel disease characterized by relapsing inflammation of the large intestine. The severity of UC is often represented by the Mayo Endoscopic Subscore (MES) which quantifies mucosal disease activity from endoscopy videos. In clinical trials, an endoscopy video is assigned an MES based upon the most severe disease activity observed in the video. For this reason, severe inflammation spread throughout the colon will receive the same MES as an otherwise healthy colon with severe inflammation restricted to a small, localized segment. Therefore, the extent of disease activity throughout the large intestine, and overall response to treatment, may not be completely captured by the MES. In this work, we aim to automatically estimate UC severity for each frame in an endoscopy video to provide a higher resolution assessment of disease activity throughout the colon. Because annotating severity at the frame-level is expensive, labor-intensive, and highly subjective, we propose a novel weakly supervised, ordinal classification method to estimate frame severity from video MES labels alone. Using clinical trial data, we first achieved 0.92 and 0.90 AUC for predicting mucosal healing and remission of UC, respectively. Then, for severity estimation, we demonstrate that our models achieve substantial Cohen's Kappa agreement with ground truth MES labels, comparable to the inter-rater agreement of expert clinicians. These findings indicate that our framework could serve as a foundation for novel clinical endpoints, based on a more localized scoring system, to better evaluate UC drug efficacy in clinical trials.