# PS-DeVCEM: Pathology-sensitive deep learning model for video capsule endoscopy based on weakly labeled data

arXiv: https://arxiv.org/abs/2011.12957 · HTML: https://mlsanity.com/paper/2011.12957/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2020.

## Built on by (most cited first, 6 of 6)

- [Automatic Estimation of Ulcerative Colitis Severity from Endoscopy Videos using Ordinal Multi-Instance Learning](https://mlsanity.com/paper/2109.14685/) (2021)
- [Evaluating clinical diversity and plausibility of synthetic capsule endoscopic images](https://mlsanity.com/paper/2301.06366/) (2023)
- [From Labels to Priors in Capsule Endoscopy: A Prior Guided Approach for Improving Generalization with Few Labels](https://mlsanity.com/paper/2206.05288/) (2022)
- [Stack of discriminative autoencoders for multiclass anomaly detection in endoscopy images](https://mlsanity.com/paper/2103.08508/) (2021)
- [Distance Metric-Based Learning with Interpolated Latent Features for Location Classification in Endoscopy Image and Video](https://mlsanity.com/paper/2103.08504/) (2021)
- [Focus-to-Perceive Representation Learning: A Cognition-Inspired Hierarchical Framework for Endoscopic Video Analysis](https://mlsanity.com/paper/2603.25778/) (2026)

## Abstract (arXiv)

We propose a novel pathology-sensitive deep learning model (PS-DeVCEM) for frame-level anomaly detection and multi-label classification of different colon diseases in video capsule endoscopy (VCE) data. Our proposed model is capable of coping with the key challenge of colon apparent heterogeneity caused by several types of diseases. Our model is driven by attention-based deep multiple instance learning and is trained end-to-end on weakly labeled data using video labels instead of detailed frame-by-frame annotation. The spatial and temporal features are obtained through ResNet50 and residual Long short-term memory (residual LSTM) blocks, respectively. Additionally, the learned temporal attention module provides the importance of each frame to the final label prediction. Moreover, we developed a self-supervision method to maximize the distance between classes of pathologies. We demonstrate through qualitative and quantitative experiments that our proposed weakly supervised learning model gives superior precision and F1-score reaching, 61.6% and 55.1%, as compared to three state-of-the-art video analysis methods respectively. We also show our model's ability to temporally localize frames with pathologies, without frame annotation information during training. Furthermore, we collected and annotated the first and largest VCE dataset with only video labels. The dataset contains 455 short video segments with 28,304 frames and 14 classes of colorectal diseases and artifacts. Dataset and code supporting this publication will be made available on our home page.