# Knowledge Augmented Relation Inference for Group Activity Recognition

arXiv: https://arxiv.org/abs/2302.14350 · HTML: https://mlsanity.com/paper/2302.14350/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 47 references that are in the map.
- First posted on arXiv 28 Feb 2023.

## Builds on (most cited first, 19 of 19)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [Skeleton-based Action Recognition via Spatial and Temporal Transformer Networks](https://mlsanity.com/paper/2008.07404/) (2020)
- [Hierarchical Deep Temporal Models for Group Activity Recognition](https://mlsanity.com/paper/1607.02643/) (2016)
- [Actor-Transformers for Group Activity Recognition](https://mlsanity.com/paper/2003.12737/) (2020)
- [Learning Actor Relation Graphs for Group Activity Recognition](https://mlsanity.com/paper/1904.10117/) (2019)
- [GroupFormer: Group Activity Recognition with Clustered Spatial-Temporal Transformer](https://mlsanity.com/paper/2108.12630/) (2021)
- [Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition](https://mlsanity.com/paper/1611.09078/) (2016)
- [Spatio-Temporal Dynamic Inference Network for Group Activity Recognition](https://mlsanity.com/paper/2108.11743/) (2021)
- [Joint Learning of Social Groups, Individuals Action and Sub-group Activities in Videos](https://mlsanity.com/paper/2007.02632/) (2020)
- [Convolutional Relational Machine for Group Activity Recognition](https://mlsanity.com/paper/1904.03308/) (2019)
- [Progressive Relation Learning for Group Activity Recognition](https://mlsanity.com/paper/1908.02948/) (2019)
- [Dual-AI: Dual-path Actor Interaction Learning for Group Activity Recognition](https://mlsanity.com/paper/2204.02148/) (2022)
- [CERN: Confidence-Energy Recurrent Network for Group Activity Recognition](https://mlsanity.com/paper/1704.03058/) (2017)
- [Detector-Free Weakly Supervised Group Activity Recognition](https://mlsanity.com/paper/2204.02139/) (2022)

## Built on by (most cited first, 2 of 2)

- [ActivityCLIP: Enhancing Group Activity Recognition by Mining Complementary Information from Text to Supplement Image Modality](https://mlsanity.com/paper/2407.19820/) (2024)
- [TrajSV: A Trajectory-based Model for Sports Video Representations and Applications](https://mlsanity.com/paper/2508.11569/) (2025)

## Abstract (arXiv)

Most existing group activity recognition methods construct spatial-temporal relations merely based on visual representation. Some methods introduce extra knowledge, such as action labels, to build semantic relations and use them to refine the visual presentation. However, the knowledge they explored just stay at the semantic-level, which is insufficient for pursing notable accuracy. In this paper, we propose to exploit knowledge concretization for the group activity recognition, and develop a novel Knowledge Augmented Relation Inference framework that can effectively use the concretized knowledge to improve the individual representations. Specifically, the framework consists of a Visual Representation Module to extract individual appearance features, a Knowledge Augmented Semantic Relation Module explore semantic representations of individual actions, and a Knowledge-Semantic-Visual Interaction Module aims to integrate visual and semantic information by the knowledge. Benefiting from these modules, the proposed framework can utilize knowledge to enhance the relation inference process and the individual representations, thus improving the performance of group activity recognition. Experimental results on two public datasets show that the proposed framework achieves competitive performance compared with state-of-the-art methods.