# Graph Convolution Neural Network For Weakly Supervised Abnormality Localization In Long Capsule Endoscopy Videos arXiv: https://arxiv.org/abs/2110.09110 ยท HTML: https://mlsanity.com/paper/2110.09110/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Builds on 34 of its 100 references that are in the map. - First posted on arXiv 18 Oct 2021. ## Builds on (most cited first, 30 of 34) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014) - [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016) - [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017) - [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017) - [Temporal Action Detection with Structured Segment Networks](https://mlsanity.com/paper/1704.06228/) (2017) - [One weird trick for parallelizing convolutional neural networks](https://mlsanity.com/paper/1404.5997/) (2014) - [The THUMOS Challenge on Action Recognition for Videos "in the Wild"](https://mlsanity.com/paper/1604.06182/) (2016) - [Graph Convolutional Networks for Temporal Action Localization](https://mlsanity.com/paper/1909.03252/) (2019) - [Weakly Supervised Deep Detection Networks](https://mlsanity.com/paper/1511.02853/) (2015) - [Multiple Instance Learning: A Survey of Problem Characteristics and Applications](https://mlsanity.com/paper/1612.03365/) (2016) - [Learning Video Object Segmentation from Static Images](https://mlsanity.com/paper/1612.02646/) (2016) - [UntrimmedNets for Weakly Supervised Action Recognition and Detection](https://mlsanity.com/paper/1703.03329/) (2017) - [Single Shot Temporal Action Detection](https://mlsanity.com/paper/1710.06236/) (2017) - [CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos](https://mlsanity.com/paper/1703.01515/) (2017) - [End-to-end Learning of Action Detection from Frame Glimpses in Videos](https://mlsanity.com/paper/1511.06984/) (2015) - [Weakly Supervised Action Localization by Sparse Temporal Pooling Network](https://mlsanity.com/paper/1712.05080/) (2017) - [W-TALC: Weakly-supervised Temporal Activity Localization and Classification](https://mlsanity.com/paper/1807.10418/) (2018) - [AutoLoc: Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/1807.08333/) (2018) - [Finding Action Tubes](https://mlsanity.com/paper/1411.6031/) (2014) - [ContextLocNet: Context-Aware Deep Network Models for Weakly Supervised Localization](https://mlsanity.com/paper/1609.04331/) (2016) - [Cascaded Boundary Regression for Temporal Action Detection](https://mlsanity.com/paper/1705.01180/) (2017) - [Automated polyp detection in colon capsule endoscopy](https://mlsanity.com/paper/1305.1912/) (2013) - [Deep Self-Taught Learning for Weakly Supervised Object Localization](https://mlsanity.com/paper/1704.05188/) (2017) - [Multiple-change-point detection for high dimensional time series via sparsified binary segmentation](https://mlsanity.com/paper/1611.08639/) (2016) - [Weakly Supervised Video Summarization by Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2001.05864/) (2020) - [Action Graphs: Weakly-supervised Action Localization with Graph Convolution Networks](https://mlsanity.com/paper/2002.01449/) (2020) - [Actionness Estimation Using Hybrid Fully Convolutional Networks](https://mlsanity.com/paper/1604.07279/) (2016) - [A Survey on Contemporary Computer-Aided Tumor, Polyp, and Ulcer Detection Methods in Wireless Capsule Endoscopy Imaging](https://mlsanity.com/paper/1910.00265/) (2019) - [Feature Selection Using Reinforcement Learning](https://mlsanity.com/paper/2101.09460/) (2021) ## Built on by (most cited first, 1 of 1) - [The intersection of video capsule endoscopy and artificial intelligence: addressing unique challenges using machine learning](https://mlsanity.com/paper/2308.13035/) (2023) ## Abstract (arXiv) Temporal activity localization in long videos is an important problem. The cost of obtaining frame level label for long Wireless Capsule Endoscopy (WCE) videos is prohibitive. In this paper, we propose an end-to-end temporal abnormality localization for long WCE videos using only weak video level labels. Physicians use Capsule Endoscopy (CE) as a non-surgical and non-invasive method to examine the entire digestive tract in order to diagnose diseases or abnormalities. While CE has revolutionized traditional endoscopy procedures, a single CE examination could last up to 8 hours generating as much as 100,000 frames. Physicians must review the entire video, frame-by-frame, in order to identify the frames capturing relevant abnormality. This, sometimes could be as few as just a single frame. Given this very high level of redundancy, analyzing long CE videos can be very tedious, time consuming and also error prone. This paper presents a novel multi-step method for an end-to-end localization of target frames capturing abnormalities of interest in the long video using only weak video labels. First we developed an automatic temporal segmentation using change point detection technique to temporally segment the video into uniform, homogeneous and identifiable segments. Then we employed Graph Convolutional Neural Network (GCNN) to learn a representation of each video segment. Using weak video segment labels, we trained our GCNN model to recognize each video segment as abnormal if it contains at least a single abnormal frame. Finally, leveraging the parameters of the trained GCNN model, we replaced the final layer of the network with a temporal pool layer to localize the relevant abnormal frames within each abnormal video segment. Our method achieved an accuracy of 89.9% on the graph classification task and a specificity of 97.5% on the abnormal frames localization task.