# Action Graphs: Weakly-supervised Action Localization with Graph Convolution Networks

arXiv: https://arxiv.org/abs/2002.01449 · HTML: https://mlsanity.com/paper/2002.01449/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2020.

## Built on by (most cited first, 16 of 16)

- [Video-Mined Task Graphs for Keystep Recognition in Instructional Videos](https://mlsanity.com/paper/2307.08763/) (2023)
- [Deep Learning-based Action Detection in Untrimmed Videos: A Survey](https://mlsanity.com/paper/2110.00111/) (2021)
- [(2.5+1)D Spatio-Temporal Scene Graphs for Video Question Answering](https://mlsanity.com/paper/2202.09277/) (2022)
- [ACGNet: Action Complement Graph Network for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2112.10977/) (2021)
- [Hierarchical Modeling for Task Recognition and Action Segmentation in Weakly-Labeled Instructional Videos](https://mlsanity.com/paper/2110.05697/) (2021)
- [A Backpack Full of Skills: Egocentric Video Understanding with Diverse Task Perspectives](https://mlsanity.com/paper/2403.03037/) (2024)
- [Deep Motion Prior for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2108.05607/) (2021)
- [DDG-Net: Discriminability-Driven Graph Network for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2307.16415/) (2023)
- [Hier-EgoPack: Hierarchical Egocentric Video Understanding with Diverse Task Perspectives](https://mlsanity.com/paper/2502.02487/) (2025)
- [Sharing Pain: Using Pain Domain Transfer for Video Recognition of Low Grade Orthopedic Pain in Horses](https://mlsanity.com/paper/2105.10313/) (2021)
- [Equine Pain Behavior Classification via Self-Supervised Disentangled Pose Representation](https://mlsanity.com/paper/2108.13258/) (2021)
- [Robust Pedestrian Attribute Recognition Using Group Sparsity for Occlusion Videos](https://mlsanity.com/paper/2110.08708/) (2021)
- [Graph Convolution Neural Network For Weakly Supervised Abnormality Localization In Long Capsule Endoscopy Videos](https://mlsanity.com/paper/2110.09110/) (2021)
- [A Generalized & Robust Framework For Timestamp Supervision in Temporal Action Segmentation](https://mlsanity.com/paper/2207.10137/) (2022)
- [Dilation-Erosion for Single-Frame Supervised Temporal Action Localization](https://mlsanity.com/paper/2212.06348/) (2022)
- [Video-Specific Query-Key Attention Modeling for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2305.04186/) (2023)

## Abstract (arXiv)

We present a method for weakly-supervised action localization based on graph convolutions. In order to find and classify video time segments that correspond to relevant action classes, a system must be able to both identify discriminative time segments in each video, and identify the full extent of each action. Achieving this with weak video level labels requires the system to use similarity and dissimilarity between moments across videos in the training data to understand both how an action appears, as well as the sub-actions that comprise the action's full extent. However, current methods do not make explicit use of similarity between video moments to inform the localization and classification predictions. We present a novel method that uses graph convolutions to explicitly model similarity between video moments. Our method utilizes similarity graphs that encode appearance and motion, and pushes the state of the art on THUMOS '14, ActivityNet 1.2, and Charades for weakly supervised action localization.