# Learning Actor Relation Graphs for Group Activity Recognition

arXiv: https://arxiv.org/abs/1904.10117 · HTML: https://mlsanity.com/paper/1904.10117/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2019.

## Built on by (most cited first, 30 of 83)

- [A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective](https://mlsanity.com/paper/2209.13232/) (2022)
- [GroupFormer: Group Activity Recognition with Clustered Spatial-Temporal Transformer](https://mlsanity.com/paper/2108.12630/) (2021)
- [Spatio-Temporal Dynamic Inference Network for Group Activity Recognition](https://mlsanity.com/paper/2108.11743/) (2021)
- [What to look at and where: Semantic and Spatial Refined Transformer for detecting human-object interactions](https://mlsanity.com/paper/2204.00746/) (2022)
- [LOGO: A Long-Form Video Dataset for Group Action Quality Assessment](https://mlsanity.com/paper/2404.05029/) (2024)
- [A Review of Deep Learning for Video Captioning](https://mlsanity.com/paper/2304.11431/) (2023)
- [Dual-AI: Dual-path Actor Interaction Learning for Group Activity Recognition](https://mlsanity.com/paper/2204.02148/) (2022)
- [Feature refinement: An expression-specific feature learning and fusion method for micro-expression recognition](https://mlsanity.com/paper/2101.04838/) (2021)
- [PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception](https://mlsanity.com/paper/2103.01933/) (2021)
- [Hunting Group Clues with Transformers for Social Group Activity Recognition](https://mlsanity.com/paper/2207.05254/) (2022)
- [Detector-Free Weakly Supervised Group Activity Recognition](https://mlsanity.com/paper/2204.02139/) (2022)
- [COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality](https://mlsanity.com/paper/2112.05892/) (2021)
- [Pose is all you need: The pose only group activity recognition system (POGARS)](https://mlsanity.com/paper/2108.04186/) (2021)
- [Neural Relational Inference with Fast Modular Meta-learning](https://mlsanity.com/paper/2310.07015/) (2023)
- [SPARTAN: Self-supervised Spatiotemporal Transformers Approach to Group Activity Recognition](https://mlsanity.com/paper/2303.12149/) (2023)
- [Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videos](https://mlsanity.com/paper/2109.07353/) (2021)
- [Panoramic Human Activity Recognition](https://mlsanity.com/paper/2203.03806/) (2022)
- [Action-slot: Visual Action-centric Representations for Multi-label Atomic Activity Recognition in Traffic Scenes](https://mlsanity.com/paper/2311.17948/) (2023)
- [A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision](https://mlsanity.com/paper/2307.03353/) (2023)
- [Towards More Practical Group Activity Detection: A New Benchmark and Model](https://mlsanity.com/paper/2312.02878/) (2023)
- [Privacy-Preserving Video Classification with Convolutional Neural Networks](https://mlsanity.com/paper/2102.03513/) (2021)
- [DROID: Driver-centric Risk Object Identification](https://mlsanity.com/paper/2106.13201/) (2021)
- [Explainable Action Prediction through Self-Supervision on Scene Graphs](https://mlsanity.com/paper/2302.03477/) (2023)
- [BiC-Net: Learning Efficient Spatio-Temporal Relation for Text-Video Retrieval](https://mlsanity.com/paper/2110.15609/) (2021)
- [Ego-Vehicle Action Recognition based on Semi-Supervised Contrastive Learning](https://mlsanity.com/paper/2303.00977/) (2023)
- [SoGAR: Self-supervised Spatiotemporal Attention-based Social Group Activity Recognition](https://mlsanity.com/paper/2305.06310/) (2023)
- [Self-supervised Social Relation Representation for Human Group Detection](https://mlsanity.com/paper/2203.03843/) (2022)
- [Entry-Flipped Transformer for Inference and Prediction of Participant Behavior](https://mlsanity.com/paper/2207.06235/) (2022)
- [Self-Correctable and Adaptable Inference for Generalizable Human Pose Estimation](https://mlsanity.com/paper/2303.11180/) (2023)
- [A Hierarchical Graph-based Approach for Recognition and Description Generation of Bimanual Actions in Videos](https://mlsanity.com/paper/2310.00670/) (2023)

## Abstract (arXiv)

Modeling relation between actors is important for recognizing group activity in a multi-person scene. This paper aims at learning discriminative relation between actors efficiently using deep models. To this end, we propose to build a flexible and efficient Actor Relation Graph (ARG) to simultaneously capture the appearance and position relation between actors. Thanks to the Graph Convolutional Network, the connections in ARG could be automatically learned from group activity videos in an end-to-end manner, and the inference on ARG could be efficiently performed with standard matrix operations. Furthermore, in practice, we come up with two variants to sparsify ARG for more effective modeling in videos: spatially localized ARG and temporal randomized ARG. We perform extensive experiments on two standard group activity recognition datasets: the Volleyball dataset and the Collective Activity dataset, where state-of-the-art performance is achieved on both datasets. We also visualize the learned actor graphs and relation features, which demonstrate that the proposed ARG is able to capture the discriminative relation information for group activity recognition.