# Real-World Graph Convolution Networks (RW-GCNs) for Action Recognition in Smart Video Surveillance

arXiv: https://arxiv.org/abs/2201.05739 · HTML: https://mlsanity.com/paper/2201.05739/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 69 references that are in the map.
- First posted on arXiv 15 Jan 2022.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [On the Relationship between Self-Attention and Convolutional Layers](https://mlsanity.com/paper/1911.03584/) (2019)
- [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019)
- [An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1902.09130/) (2019)
- [STM: SpatioTemporal and Motion Encoding for Action Recognition](https://mlsanity.com/paper/1908.02486/) (2019)
- [Cross-view Action Modeling, Learning and Recognition](https://mlsanity.com/paper/1405.2941/) (2014)
- [Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks](https://mlsanity.com/paper/2105.02358/) (2021)
- [Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition](https://mlsanity.com/paper/2010.09978/) (2020)
- [Multi-granularity Generator for Temporal Action Proposal](https://mlsanity.com/paper/1811.11524/) (2018)
- [Spatial Temporal Transformer Network for Skeleton-based Action Recognition](https://mlsanity.com/paper/2012.06399/) (2020)
- [Learning Implicitly Recurrent CNNs Through Parameter Sharing](https://mlsanity.com/paper/1902.09701/) (2019)
- [Scalable Mutual Information Estimation using Dependence Graphs](https://mlsanity.com/paper/1801.09125/) (2018)
- [Feedback Graph Convolutional Network for Skeleton-based Action Recognition](https://mlsanity.com/paper/2003.07564/) (2020)
- [Decoupled Spatial-Temporal Attention Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2007.03263/) (2020)
- [Convolutional Neural Networks with Alternately Updated Clique](https://mlsanity.com/paper/1802.10419/) (2018)

## Built on by (most cited first, 6 of 6)

- [Semantic2Graph: Graph-based Multi-modal Feature Fusion for Action Segmentation in Videos](https://mlsanity.com/paper/2209.05653/) (2022)
- [Pishgu: Universal Path Prediction Network Architecture for Real-time Cyber-physical Edge Systems](https://mlsanity.com/paper/2210.08057/) (2022)
- [Understanding Ethics, Privacy, and Regulations in Smart Video Surveillance for Public Safety](https://mlsanity.com/paper/2212.12936/) (2022)
- [ADG-Pose: Automated Dataset Generation for Real-World Human Pose Estimation](https://mlsanity.com/paper/2202.00753/) (2022)
- [Action Recognition in Real-World Ambient Assisted Living Environment](https://mlsanity.com/paper/2503.23214/) (2025)
- [Adversarially-Refined VQ-GAN with Dense Motion Tokenization for Spatio-Temporal Heatmaps](https://mlsanity.com/paper/2509.19252/) (2025)

## Abstract (arXiv)

Action recognition is a key algorithmic part of emerging on-the-edge smart video surveillance and security systems. Skeleton-based action recognition is an attractive approach which, instead of using RGB pixel data, relies on human pose information to classify appropriate actions. However, existing algorithms often assume ideal conditions that are not representative of real-world limitations, such as noisy input, latency requirements, and edge resource constraints. To address the limitations of existing approaches, this paper presents Real-World Graph Convolution Networks (RW-GCNs), an architecture-level solution for meeting the domain constraints of Real World Skeleton-based Action Recognition. Inspired by the presence of feedback connections in the human visual cortex, RW-GCNs leverage attentive feedback augmentation on existing near state-of-the-art (SotA) Spatial-Temporal Graph Convolution Networks (ST-GCNs). The ST-GCNs' design choices are derived from information theory-centric principles to address both the spatial and temporal noise typically encountered in end-to-end real-time and on-the-edge smart video systems. Our results demonstrate RW-GCNs' ability to serve these applications by achieving a new SotA accuracy on the NTU-RGB-D-120 dataset at 94.1%, and achieving 32X less latency than baseline ST-GCN applications while still achieving 90.4% accuracy on the Northwestern UCLA dataset in the presence of spatial keypoint noise. RW-GCNs further show system scalability by running on the 10X cost effective NVIDIA Jetson Nano (as opposed to NVIDIA Xavier NX), while still maintaining a respectful range of throughput (15.6 to 5.5 Actions per Second) on the resource constrained device. The code is available here: https://github.com/TeCSAR-UNCC/RW-GCN.