Graph Convolution Neural Network For Weakly Supervised Abnormality Localization In Long Capsule Endoscopy Videos
Not on the map. Why?
Abstract (arXiv)
Temporal activity localization in long videos is an important problem. The cost of obtaining frame level label for long Wireless Capsule Endoscopy (WCE) videos is prohibitive. In this paper, we propose an end-to-end temporal abnormality localization for long WCE videos using only weak video level labels. Physicians use Capsule Endoscopy (CE) as a non-surgical and non-invasive method to examine the entire digestive tract in order to diagnose diseases or abnormalities. While CE has revolutionized traditional endoscopy procedures, a single CE examination could last up to 8 hours generating as much as 100,000 frames. Physicians must review the entire video, frame-by-frame, in order to identify the frames capturing relevant abnormality. This, sometimes could be as few as just a single frame. Given this very high level of redundancy, analyzing long CE videos can be very tedious, time consuming and also error prone. This paper presents a novel multi-step method for an end-to-end localization of target frames capturing abnormalities of interest in the long video using only weak video labels. First we developed an automatic temporal segmentation using change point detection technique to temporally segment the video into uniform, homogeneous and identifiable segments. Then we employed Graph Convolutional Neural Network (GCNN) to learn a representation of each video segment. Using weak video segment labels, we trained our GCNN model to recognize each video segment as abnormal if it contains at least a single abnormal frame. Finally, leveraging the parameters of the trained GCNN model, we replaced the final layer of the network with a temporal pool layer to localize the relevant abnormal frames within each abnormal video segment. Our method achieved an accuracy of 89.9% on the graph classification task and a specificity of 97.5% on the abnormal frames localization task.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 100 references that are in the map. First posted on arXiv 18 Oct 2021.
Builds on
34 of 100 references.
References · 34
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2017Graph Attention Networkscited by 8,854
- 2017Inductive Representation Learning on Large Graphscited by 6,284
- 2017Temporal Action Detection with Structured Segment Networkscited by 265
- 2014One weird trick for parallelizing convolutional neural networkscited by 264
- 2016The THUMOS Challenge on Action Recognition for Videos "in the Wild"cited by 213
- 2019Graph Convolutional Networks for Temporal Action Localizationcited by 208
- 2015Weakly Supervised Deep Detection Networkscited by 183
- 2016Multiple Instance Learning: A Survey of Problem Characteristics and Applicationscited by 163
- 2016Learning Video Object Segmentation from Static Imagescited by 151
22 more
- 2017UntrimmedNets for Weakly Supervised Action Recognition and Detectioncited by 129
- 2017Single Shot Temporal Action Detectioncited by 123
- 2017CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videoscited by 112
- 2015End-to-end Learning of Action Detection from Frame Glimpses in Videoscited by 101
- 2017Weakly Supervised Action Localization by Sparse Temporal Pooling Networkcited by 100
- 2018W-TALC: Weakly-supervised Temporal Activity Localization and Classificationcited by 87
- 2018AutoLoc: Weakly-supervised Temporal Action Localizationcited by 75
- 2014Finding Action Tubescited by 50
- 2016ContextLocNet: Context-Aware Deep Network Models for Weakly Supervised Localizationcited by 39
- 2017Cascaded Boundary Regression for Temporal Action Detectioncited by 35
- 2013Automated polyp detection in colon capsule endoscopycited by 30
- 2017Deep Self-Taught Learning for Weakly Supervised Object Localizationcited by 27
- 2016Multiple-change-point detection for high dimensional time series via sparsified binary segmentationcited by 24
- 2020Weakly Supervised Video Summarization by Hierarchical Reinforcement Learningcited by 18
- 2020Action Graphs: Weakly-supervised Action Localization with Graph Convolution Networkscited by 16
- 2016Actionness Estimation Using Hybrid Fully Convolutional Networkscited by 15
- 2019A Survey on Contemporary Computer-Aided Tumor, Polyp, and Ulcer Detection Methods in Wireless Capsule Endoscopy Imagingcited by 7
- 2021Feature Selection Using Reinforcement Learningcited by 6
- 2020Hierarchical Deep Convolutional Neural Networks for Multi-category Diagnosis of Gastrointestinal Disorders on Histopathological Imagescited by 5
- 2017ActivityNet Challenge 2017 Summarycited by 4
- 2020Dialogue-Based Simulation For Cultural Awareness Trainingcited by 4
- 2021Lesion2Vec: Deep Metric Learning for Few-Shot Multiple Lesions Recognition in Wireless Capsule Endoscopy Videocited by 3
Built on by
Citing papers per year: 2023: 1.
1 paper.