# Graph Convolutional Module for Temporal Action Localization in Videos

arXiv: https://arxiv.org/abs/2112.00302 · HTML: https://mlsanity.com/paper/2112.00302/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 56 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling](https://mlsanity.com/paper/1801.10247/) (2018)
- [BMN: Boundary-Matching Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1907.09702/) (2019)
- [Relation Networks for Object Detection](https://mlsanity.com/paper/1711.11575/) (2017)
- [BSN: Boundary Sensitive Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1806.02964/) (2018)
- [Temporal Action Detection with Structured Segment Networks](https://mlsanity.com/paper/1704.06228/) (2017)
- [G-TAD: Sub-Graph Localization for Temporal Action Detection](https://mlsanity.com/paper/1911.11462/) (2019)
- [Videos as Space-Time Region Graphs](https://mlsanity.com/paper/1806.01810/) (2018)
- [Graph Convolutional Networks for Temporal Action Localization](https://mlsanity.com/paper/1909.03252/) (2019)
- [Rethinking the Faster R-CNN Architecture for Temporal Action Localization](https://mlsanity.com/paper/1804.07667/) (2018)
- [Adaptive Sampling Towards Fast Graph Representation Learning](https://mlsanity.com/paper/1809.05343/) (2018)
- [R-C3D: Region Convolutional 3D Network for Temporal Activity Detection](https://mlsanity.com/paper/1703.07814/) (2017)
- [UntrimmedNets for Weakly Supervised Action Recognition and Detection](https://mlsanity.com/paper/1703.03329/) (2017)
- [Single Shot Temporal Action Detection](https://mlsanity.com/paper/1710.06236/) (2017)
- [TURN TAP: Temporal Unit Regression Network for Temporal Action Proposals](https://mlsanity.com/paper/1703.06189/) (2017)
- [CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos](https://mlsanity.com/paper/1703.01515/) (2017)
- [End-to-end Learning of Action Detection from Frame Glimpses in Videos](https://mlsanity.com/paper/1511.06984/) (2015)
- [CUHK & ETHZ & SIAT Submission to ActivityNet Challenge 2016](https://mlsanity.com/paper/1608.00797/) (2016)
- [CTAP: Complementary Temporal Action Proposal Generation](https://mlsanity.com/paper/1807.04821/) (2018)
- [End-to-End Learning of Motion Representation for Video Understanding](https://mlsanity.com/paper/1804.00413/) (2018)
- [Temporal Gaussian Mixture Layer for Videos](https://mlsanity.com/paper/1803.06316/) (2018)

## Built on by (most cited first, 29 of 29)

- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [Colar: Effective and Efficient Online Action Detection by Consulting Exemplars](https://mlsanity.com/paper/2203.01057/) (2022)
- [Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2403.11491/) (2024)
- [Learning to Refactor Action and Co-occurrence Features for Temporal Action Localization](https://mlsanity.com/paper/2206.11493/) (2022)
- [Benchmarking the Robustness of Temporal Action Detection Models Against Temporal Corruptions](https://mlsanity.com/paper/2403.20254/) (2024)
- [Semantic2Graph: Graph-based Multi-modal Feature Fusion for Action Segmentation in Videos](https://mlsanity.com/paper/2209.05653/) (2022)
- [Structured Attention Composition for Temporal Action Localization](https://mlsanity.com/paper/2205.09956/) (2022)
- [Self-Supervised Representation Learning with Spatial-Temporal Consistency for Sign Language Recognition](https://mlsanity.com/paper/2406.10501/) (2024)
- [Towards Long Video Understanding via Fine-detailed Video Story Generation](https://mlsanity.com/paper/2412.06182/) (2024)
- [Temporal Action Detection Model Compression by Progressive Block Drop](https://mlsanity.com/paper/2503.16916/) (2025)
- [Spiking Variational Graph Representation Inference for Video Summarization](https://mlsanity.com/paper/2508.15389/) (2025)
- [Weakly-Supervised Action Localization by Hierarchically-structured Latent Attention Modeling](https://mlsanity.com/paper/2308.09946/) (2023)
- [UniAV: Unified Audio-Visual Perception for Multi-Task Video Event Localization](https://mlsanity.com/paper/2404.03179/) (2024)
- [Exploring Audio Cues for Enhanced Test-Time Video Model Adaptation](https://mlsanity.com/paper/2506.12481/) (2025)
- [Language-Guided Graph Representation Learning for Video Summarization](https://mlsanity.com/paper/2511.10953/) (2025)
- [Proposal-based Temporal Action Localization with Point-level Supervision](https://mlsanity.com/paper/2310.05511/) (2023)
- [A Hybrid Graph Network for Complex Activity Detection in Video](https://mlsanity.com/paper/2310.17493/) (2023)
- [CFSSeg: Closed-Form Solution for Class-Incremental Semantic Segmentation of 2D Images and 3D Point Clouds](https://mlsanity.com/paper/2412.10834/) (2024)
- [SigFormer: Sparse Signal-Guided Transformer for Multi-Modal Human Action Segmentation](https://mlsanity.com/paper/2311.17428/) (2023)
- [Boosting Semi-Supervised Temporal Action Localization by Learning from Non-Target Classes](https://mlsanity.com/paper/2403.11189/) (2024)
- [ECGDeDRDNet: A deep learning-based method for Electrocardiogram noise removal using a double recurrent dense network](https://mlsanity.com/paper/2505.05477/) (2025)
- [EgoLoc: A Generalizable Solution for Temporal Interaction Localization in Egocentric Videos](https://mlsanity.com/paper/2508.12349/) (2025)
- [Boundary-Aware Proposal Generation Method for Temporal Action Localization](https://mlsanity.com/paper/2309.13810/) (2023)
- [FMI-TAL: Few-shot Multiple Instances Temporal Action Localization by Probability Distribution Learning and Interval Cluster Refinement](https://mlsanity.com/paper/2408.13765/) (2024)
- [ProTAL: A Drag-and-Link Video Programming Framework for Temporal Action Localization](https://mlsanity.com/paper/2505.17555/) (2025)
- [Generative Model-Based Feature Attention Module for Video Action Analysis](https://mlsanity.com/paper/2508.13565/) (2025)
- [Language-guided Recursive Spatiotemporal Graph Modeling for Video Summarization](https://mlsanity.com/paper/2509.05604/) (2025)
- [Boosting Point-supervised Temporal Action Localization via Text Refinement and Alignment](https://mlsanity.com/paper/2602.01257/) (2026)
- [Exploring the Temporal Consistency for Point-Level Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2602.05718/) (2026)

## Abstract (arXiv)

Temporal action localization has long been researched in computer vision. Existing state-of-the-art action localization methods divide each video into multiple action units (i.e., proposals in two-stage methods and segments in one-stage methods) and then perform action recognition/regression on each of them individually, without explicitly exploiting their relations during learning. In this paper, we claim that the relations between action units play an important role in action localization, and a more powerful action detector should not only capture the local content of each action unit but also allow a wider field of view on the context related to it. To this end, we propose a general graph convolutional module (GCM) that can be easily plugged into existing action localization methods, including two-stage and one-stage paradigms. To be specific, we first construct a graph, where each action unit is represented as a node and their relations between two action units as an edge. Here, we use two types of relations, one for capturing the temporal connections between different action units, and the other one for characterizing their semantic relationship. Particularly for the temporal connections in two-stage methods, we further explore two different kinds of edges, one connecting the overlapping action units and the other one connecting surrounding but disjointed units. Upon the graph we built, we then apply graph convolutional networks (GCNs) to model the relations among different action units, which is able to learn more informative representations to enhance action localization. Experimental results show that our GCM consistently improves the performance of existing action localization methods, including two-stage methods (e.g., CBR and R-C3D) and one-stage methods (e.g., D-SSAD), verifying the generality and effectiveness of our GCM.