# Detector-Free Weakly Supervised Group Activity Recognition

arXiv: https://arxiv.org/abs/2204.02139 · HTML: https://mlsanity.com/paper/2204.02139/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 65 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [HOTR: End-to-End Human-Object Interaction Detection with Transformers](https://mlsanity.com/paper/2104.13682/) (2021)
- [End-to-End Human Object Interaction Detection with HOI Transformer](https://mlsanity.com/paper/2103.04503/) (2021)
- [Hierarchical Deep Temporal Models for Group Activity Recognition](https://mlsanity.com/paper/1607.02643/) (2016)
- [Actor-Transformers for Group Activity Recognition](https://mlsanity.com/paper/2003.12737/) (2020)
- [Learning Actor Relation Graphs for Group Activity Recognition](https://mlsanity.com/paper/1904.10117/) (2019)
- [Video Modeling with Correlation Networks](https://mlsanity.com/paper/1906.03349/) (2019)
- [MotionSqueeze: Neural Motion Feature Learning for Video Understanding](https://mlsanity.com/paper/2007.09933/) (2020)
- [GroupFormer: Group Activity Recognition with Clustered Spatial-Temporal Transformer](https://mlsanity.com/paper/2108.12630/) (2021)
- [Representation Flow for Action Recognition](https://mlsanity.com/paper/1810.01455/) (2018)
- [Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition](https://mlsanity.com/paper/1611.09078/) (2016)
- [End-to-End Learning of Motion Representation for Video Understanding](https://mlsanity.com/paper/1804.00413/) (2018)

## Built on by (most cited first, 20 of 20)

- [Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals](https://mlsanity.com/paper/2412.09758/) (2024)
- [SPARTAN: Self-supervised Spatiotemporal Transformers Approach to Group Activity Recognition](https://mlsanity.com/paper/2303.12149/) (2023)
- [Causal Temporal Representation Learning with Nonstationary Sparse Transition](https://mlsanity.com/paper/2409.03142/) (2024)
- [A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision](https://mlsanity.com/paper/2307.03353/) (2023)
- [Towards More Practical Group Activity Detection: A New Benchmark and Model](https://mlsanity.com/paper/2312.02878/) (2023)
- [SoGAR: Self-supervised Spatiotemporal Attention-based Social Group Activity Recognition](https://mlsanity.com/paper/2305.06310/) (2023)
- [GPT4Ego: Unleashing the Potential of Pre-trained Models for Zero-Shot Egocentric Action Recognition](https://mlsanity.com/paper/2401.10039/) (2024)
- [Skeleton-based Group Activity Recognition via Spatial-Temporal Panoramic Graph](https://mlsanity.com/paper/2407.19497/) (2024)
- [REACT: Recognize Every Action Everywhere All At Once](https://mlsanity.com/paper/2312.00188/) (2023)
- [Flow-Assisted Motion Learning Network for Weakly-Supervised Group Activity Recognition](https://mlsanity.com/paper/2405.18012/) (2024)
- [Knowledge Augmented Relation Inference for Group Activity Recognition](https://mlsanity.com/paper/2302.14350/) (2023)
- [Learning Group Activity Features Through Person Attribute Prediction](https://mlsanity.com/paper/2403.02753/) (2024)
- [Self-supervised Multi-actor Social Activity Understanding in Streaming Videos](https://mlsanity.com/paper/2406.14472/) (2024)
- [Spatio-Temporal Proximity-Aware Dual-Path Model for Panoramic Activity Recognition](https://mlsanity.com/paper/2403.14113/) (2024)
- [LiGAR: LiDAR-Guided Hierarchical Transformer for Multi-Modal Group Activity Recognition](https://mlsanity.com/paper/2410.21108/) (2024)
- [MPT-PAR:Mix-Parameters Transformer for Panoramic Activity Recognition](https://mlsanity.com/paper/2408.00420/) (2024)
- [PromptGAR: Flexible Promptive Group Activity Recognition](https://mlsanity.com/paper/2503.08933/) (2025)
- [Part-Aware Bottom-Up Group Reasoning for Fine-Grained Social Interaction Detection](https://mlsanity.com/paper/2511.03666/) (2025)
- [Human-in-the-loop Adaptation in Group Activity Feature Learning for Team Sports Video Retrieval](https://mlsanity.com/paper/2602.03157/) (2026)
- [Group-DINOmics: Incorporating People Dynamics into DINO for Self-supervised Group Activity Feature Learning](https://mlsanity.com/paper/2604.04467/) (2026)

## Abstract (arXiv)

Group activity recognition is the task of understanding the activity conducted by a group of people as a whole in a multi-person video. Existing models for this task are often impractical in that they demand ground-truth bounding box labels of actors even in testing or rely on off-the-shelf object detectors. Motivated by this, we propose a novel model for group activity recognition that depends neither on bounding box labels nor on object detector. Our model based on Transformer localizes and encodes partial contexts of a group activity by leveraging the attention mechanism, and represents a video clip as a set of partial context embeddings. The embedding vectors are then aggregated to form a single group representation that reflects the entire context of an activity while capturing temporal evolution of each partial context. Our method achieves outstanding performance on two benchmarks, Volleyball and NBA datasets, surpassing not only the state of the art trained with the same level of supervision, but also some of existing models relying on stronger supervision.