# Progressive Relation Learning for Group Activity Recognition

arXiv: https://arxiv.org/abs/1908.02948 · HTML: https://mlsanity.com/paper/1908.02948/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2019.

## Built on by (most cited first, 28 of 28)

- [GroupFormer: Group Activity Recognition with Clustered Spatial-Temporal Transformer](https://mlsanity.com/paper/2108.12630/) (2021)
- [Spatio-Temporal Dynamic Inference Network for Group Activity Recognition](https://mlsanity.com/paper/2108.11743/) (2021)
- [Dual-AI: Dual-path Actor Interaction Learning for Group Activity Recognition](https://mlsanity.com/paper/2204.02148/) (2022)
- [Hunting Group Clues with Transformers for Social Group Activity Recognition](https://mlsanity.com/paper/2207.05254/) (2022)
- [Detector-Free Weakly Supervised Group Activity Recognition](https://mlsanity.com/paper/2204.02139/) (2022)
- [COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality](https://mlsanity.com/paper/2112.05892/) (2021)
- [Learning to Recommend Frame for Interactive Video Object Segmentation in the Wild](https://mlsanity.com/paper/2103.10391/) (2021)
- [SPARTAN: Self-supervised Spatiotemporal Transformers Approach to Group Activity Recognition](https://mlsanity.com/paper/2303.12149/) (2023)
- [A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision](https://mlsanity.com/paper/2307.03353/) (2023)
- [Towards More Practical Group Activity Detection: A New Benchmark and Model](https://mlsanity.com/paper/2312.02878/) (2023)
- [A Two-stream Neural Network for Pose-based Hand Gesture Recognition](https://mlsanity.com/paper/2101.08926/) (2021)
- [SoGAR: Self-supervised Spatiotemporal Attention-based Social Group Activity Recognition](https://mlsanity.com/paper/2305.06310/) (2023)
- [Skeleton-based Group Activity Recognition via Spatial-Temporal Panoramic Graph](https://mlsanity.com/paper/2407.19497/) (2024)
- [Spatiotemporal Deformable Scene Graphs for Complex Activity Detection](https://mlsanity.com/paper/2104.08194/) (2021)
- [REACT: Recognize Every Action Everywhere All At Once](https://mlsanity.com/paper/2312.00188/) (2023)
- [Learning Group Activities from Skeletons without Individual Action Labels](https://mlsanity.com/paper/2105.06754/) (2021)
- [Flow-Assisted Motion Learning Network for Weakly-Supervised Group Activity Recognition](https://mlsanity.com/paper/2405.18012/) (2024)
- [Knowledge Augmented Relation Inference for Group Activity Recognition](https://mlsanity.com/paper/2302.14350/) (2023)
- [DECOMPL: Decompositional Learning with Attention Pooling for Group Activity Recognition from a Single Volleyball Image](https://mlsanity.com/paper/2303.06439/) (2023)
- [A Hybrid Graph Network for Complex Activity Detection in Video](https://mlsanity.com/paper/2310.17493/) (2023)
- [MLP-AIR: An Efficient MLP-Based Method for Actor Interaction Relation Learning in Group Activity Recognition](https://mlsanity.com/paper/2304.08803/) (2023)
- [Spatio-Temporal Proximity-Aware Dual-Path Model for Panoramic Activity Recognition](https://mlsanity.com/paper/2403.14113/) (2024)
- [Design and Analysis of Efficient Attention in Transformers for Social Group Activity Recognition](https://mlsanity.com/paper/2404.09964/) (2024)
- [Group Activity Recognition via Dynamic Composition and Interaction](https://mlsanity.com/paper/2305.05583/) (2023)
- [Group Activity Recognition in Computer Vision: A Comprehensive Review, Challenges, and Future Perspectives](https://mlsanity.com/paper/2307.13541/) (2023)
- [ActivityCLIP: Enhancing Group Activity Recognition by Mining Complementary Information from Text to Supplement Image Modality](https://mlsanity.com/paper/2407.19820/) (2024)
- [VicKAM: Visual Conceptual Knowledge Guided Action Map for Weakly Supervised Group Activity Recognition](https://mlsanity.com/paper/2502.09967/) (2025)
- [BasketEvent: Understanding Who Did What and When in Basketball Videos](https://mlsanity.com/paper/2607.21267/) (2026)

## Abstract (arXiv)

Group activities usually involve spatiotemporal dynamics among many interactive individuals, while only a few participants at several key frames essentially define the activity. Therefore, effectively modeling the group-relevant and suppressing the irrelevant actions (and interactions) are vital for group activity recognition. In this paper, we propose a novel method based on deep reinforcement learning to progressively refine the low-level features and high-level relations of group activities. Firstly, we construct a semantic relation graph (SRG) to explicitly model the relations among persons. Then, two agents adopting policy according to two Markov decision processes are applied to progressively refine the SRG. Specifically, one feature-distilling (FD) agent in the discrete action space refines the low-level spatio-temporal features by distilling the most informative frames. Another relation-gating (RG) agent in continuous action space adjusts the high-level semantic graph to pay more attention to group-relevant relations. The SRG, FD agent, and RG agent are optimized alternately to mutually boost the performance of each other. Extensive experiments on two widely used benchmarks demonstrate the effectiveness and superiority of the proposed approach.