# Panoramic Human Activity Recognition

arXiv: https://arxiv.org/abs/2203.03806 · HTML: https://mlsanity.com/paper/2203.03806/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 57 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 25 of 25)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [Actor-Context-Actor Relation Network for Spatio-Temporal Action Localization](https://mlsanity.com/paper/2006.07976/) (2020)
- [Hierarchical Deep Temporal Models for Group Activity Recognition](https://mlsanity.com/paper/1607.02643/) (2016)
- [Actor-Transformers for Group Activity Recognition](https://mlsanity.com/paper/2003.12737/) (2020)
- [MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actions](https://mlsanity.com/paper/2105.07404/) (2021)
- [Learning Actor Relation Graphs for Group Activity Recognition](https://mlsanity.com/paper/1904.10117/) (2019)
- [Asynchronous Interaction Aggregation for Action Detection](https://mlsanity.com/paper/2004.07485/) (2020)
- [SF-Net: Single-Frame Supervision for Temporal Action Localization](https://mlsanity.com/paper/2003.06845/) (2020)
- [Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning](https://mlsanity.com/paper/1805.02335/) (2018)
- [PANDA: A Gigapixel-level Human-centric Video Dataset](https://mlsanity.com/paper/2003.04852/) (2020)
- [Deep Learning on Lie Groups for Skeleton-based Action Recognition](https://mlsanity.com/paper/1612.05877/) (2016)
- [Joint Learning of Social Groups, Individuals Action and Sub-group Activities in Videos](https://mlsanity.com/paper/2007.02632/) (2020)
- [JRDB-Act: A Large-scale Dataset for Spatio-temporal Action, Social Group and Activity Detection](https://mlsanity.com/paper/2106.08827/) (2021)
- [Spatio-Temporal Channel Correlation Networks for Action Classification](https://mlsanity.com/paper/1806.07754/) (2018)
- [Context-Aware RCNN: A Baseline for Action Detection in Videos](https://mlsanity.com/paper/2007.09861/) (2020)
- [CERN: Confidence-Energy Recurrent Network for Group Activity Recognition](https://mlsanity.com/paper/1704.03058/) (2017)
- [Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition](https://mlsanity.com/paper/1809.01407/) (2018)
- [Spot On: Action Localization from Pointly-Supervised Proposals](https://mlsanity.com/paper/1604.07602/) (2016)
- [GD-GAN: Generative Adversarial Networks for Trajectory Prediction and Group Detection in Crowds](https://mlsanity.com/paper/1812.07667/) (2018)
- [Human Action Localization with Sparse Spatial Supervision](https://mlsanity.com/paper/1605.05197/) (2016)
- [Interaction-aware Spatio-temporal Pyramid Attention Networks for Action Classification](https://mlsanity.com/paper/1808.01106/) (2018)

## Built on by (most cited first, 13 of 13)

- [Omnidirectional Multi-Object Tracking](https://mlsanity.com/paper/2503.04565/) (2025)
- [Towards More Practical Group Activity Detection: A New Benchmark and Model](https://mlsanity.com/paper/2312.02878/) (2023)
- [SoGAR: Self-supervised Spatiotemporal Attention-based Social Group Activity Recognition](https://mlsanity.com/paper/2305.06310/) (2023)
- [REACT: Recognize Every Action Everywhere All At Once](https://mlsanity.com/paper/2312.00188/) (2023)
- [Real-time Trajectory-based Social Group Detection](https://mlsanity.com/paper/2304.05678/) (2023)
- [Dynamic Group Detection using VLM-augmented Temporal Groupness Graph](https://mlsanity.com/paper/2509.04758/) (2025)
- [OmniTrack++: Omnidirectional Multi-Object Tracking by Learning Large-FoV Trajectory Feedback](https://mlsanity.com/paper/2511.00510/) (2025)
- [Boundary Discretization and Reliable Classification Network for Temporal Action Detection](https://mlsanity.com/paper/2310.06403/) (2023)
- [Spatio-Temporal Proximity-Aware Dual-Path Model for Panoramic Activity Recognition](https://mlsanity.com/paper/2403.14113/) (2024)
- [LiGAR: LiDAR-Guided Hierarchical Transformer for Multi-Modal Group Activity Recognition](https://mlsanity.com/paper/2410.21108/) (2024)
- [Structured Relational Reasoning for Group Activity Assessment](https://mlsanity.com/paper/2508.07996/) (2025)
- [Social-MAE: Social Masked Autoencoder for Multi-person Motion Representation Learning](https://mlsanity.com/paper/2404.05578/) (2024)
- [MPT-PAR:Mix-Parameters Transformer for Panoramic Activity Recognition](https://mlsanity.com/paper/2408.00420/) (2024)

## Abstract (arXiv)

To obtain a more comprehensive activity understanding for a crowded scene, in this paper, we propose a new problem of panoramic human activity recognition (PAR), which aims to simultaneous achieve the individual action, social group activity, and global activity recognition. This is a challenging yet practical problem in real-world applications. For this problem, we develop a novel hierarchical graph neural network to progressively represent and model the multi-granularity human activities and mutual social relations for a crowd of people. We further build a benchmark to evaluate the proposed method and other existing related methods. Experimental results verify the rationality of the proposed PAR problem, the effectiveness of our method and the usefulness of the benchmark. We will release the source code and benchmark to the public for promoting the study on this problem.