# Joint Inference of Groups, Events and Human Roles in Aerial Videos

arXiv: https://arxiv.org/abs/1505.05957 · HTML: https://mlsanity.com/paper/1505.05957/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2015.

## Built on by (most cited first, 30 of 30)

- [OadTR: Online Action Detection with Transformers](https://mlsanity.com/paper/2106.11149/) (2021)
- [GroupFormer: Group Activity Recognition with Clustered Spatial-Temporal Transformer](https://mlsanity.com/paper/2108.12630/) (2021)
- [AGENT: A Benchmark for Core Psychological Reasoning](https://mlsanity.com/paper/2102.12321/) (2021)
- [LOGO: A Long-Form Video Dataset for Group Action Quality Assessment](https://mlsanity.com/paper/2404.05029/) (2024)
- [Dual-AI: Dual-path Actor Interaction Learning for Group Activity Recognition](https://mlsanity.com/paper/2204.02148/) (2022)
- [PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception](https://mlsanity.com/paper/2103.01933/) (2021)
- [Hunting Group Clues with Transformers for Social Group Activity Recognition](https://mlsanity.com/paper/2207.05254/) (2022)
- [Detector-Free Weakly Supervised Group Activity Recognition](https://mlsanity.com/paper/2204.02139/) (2022)
- [Multi-modal Prompting for Low-Shot Temporal Action Localization](https://mlsanity.com/paper/2303.11732/) (2023)
- [RingMo-Aerial: An Aerial Remote Sensing Foundation Model With Affine Transformation Contrastive Learning](https://mlsanity.com/paper/2409.13366/) (2024)
- [Constraint and Union for Partially-Supervised Temporal Sentence Grounding](https://mlsanity.com/paper/2302.09850/) (2023)
- [SPARTAN: Self-supervised Spatiotemporal Transformers Approach to Group Activity Recognition](https://mlsanity.com/paper/2303.12149/) (2023)
- [Context-Enhanced Memory-Refined Transformer for Online Action Detection](https://mlsanity.com/paper/2503.18359/) (2025)
- [Distilling Vision-Language Pre-training to Collaborate with Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2212.09335/) (2022)
- [SoGAR: Self-supervised Spatiotemporal Attention-based Social Group Activity Recognition](https://mlsanity.com/paper/2305.06310/) (2023)
- [Learning to Discriminate Information for Online Action Detection: Analysis and Application](https://mlsanity.com/paper/2109.03393/) (2021)
- [Skeleton-based Group Activity Recognition via Spatial-Temporal Panoramic Graph](https://mlsanity.com/paper/2407.19497/) (2024)
- [REACT: Recognize Every Action Everywhere All At Once](https://mlsanity.com/paper/2312.00188/) (2023)
- [Overcoming Multi-step Complexity in Multimodal Theory-of-Mind Reasoning: A Scalable Bayesian Planner](https://mlsanity.com/paper/2506.01301/) (2025)
- [HAtt-Flow: Hierarchical Attention-Flow Mechanism for Group Activity Scene Graph Generation in Videos](https://mlsanity.com/paper/2312.07740/) (2023)
- [Flow-Assisted Motion Learning Network for Weakly-Supervised Group Activity Recognition](https://mlsanity.com/paper/2405.18012/) (2024)
- [Contrast-Unity for Partially-Supervised Temporal Sentence Grounding](https://mlsanity.com/paper/2502.12917/) (2025)
- [Interaction-GCN: A Graph Convolutional Network based framework for social interaction recognition in egocentric videos](https://mlsanity.com/paper/2104.14007/) (2021)
- [FuTH-Net: Fusing Temporal Relations and Holistic Features for Aerial Video Classification](https://mlsanity.com/paper/2209.11316/) (2022)
- [Neural Force Field: Few-shot Learning of Generalized Physical Reasoning](https://mlsanity.com/paper/2502.08987/) (2025)
- [A Dynamic Data Driven Approach for Explainable Scene Understanding](https://mlsanity.com/paper/2206.09089/) (2022)
- [VicKAM: Visual Conceptual Knowledge Guided Action Map for Weakly Supervised Group Activity Recognition](https://mlsanity.com/paper/2502.09967/) (2025)
- [PromptGAR: Flexible Promptive Group Activity Recognition](https://mlsanity.com/paper/2503.08933/) (2025)
- [Trajectory prediction for heterogeneous agents: A performance analysis on small and imbalanced datasets](https://mlsanity.com/paper/2510.03776/) (2025)
- [SED:Lightweight Saliency prediction for Event-based data via Distillation](https://mlsanity.com/paper/2606.14631/) (2026)

## Abstract (arXiv)

With the advent of drones, aerial video analysis becomes increasingly important; yet, it has received scant attention in the literature. This paper addresses a new problem of parsing low-resolution aerial videos of large spatial areas, in terms of 1) grouping, 2) recognizing events and 3) assigning roles to people engaged in events. We propose a novel framework aimed at conducting joint inference of the above tasks, as reasoning about each in isolation typically fails in our setting. Given noisy tracklets of people and detections of large objects and scene surfaces (e.g., building, grass), we use a spatiotemporal AND-OR graph to drive our joint inference, using Markov Chain Monte Carlo and dynamic programming. We also introduce a new formalism of spatiotemporal templates characterizing latent sub-events. For evaluation, we have collected and released a new aerial videos dataset using a hex-rotor flying over picnic areas rich with group events. Our results demonstrate that we successfully address above inference tasks under challenging conditions.