# Pixels or Positions? Benchmarking Modalities in Group Activity Recognition arXiv: https://arxiv.org/abs/2511.12606 · HTML: https://mlsanity.com/paper/2511.12606/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Builds on 31 of its 53 references that are in the map. - First posted on arXiv 16 Nov 2025. ## Builds on (most cited first, 30 of 31) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016) - [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017) - [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018) - [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025) - [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022) - [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018) - [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021) - [VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking](https://mlsanity.com/paper/2303.16727/) (2023) - [DeepGCNs: Can GCNs Go as Deep as CNNs?](https://mlsanity.com/paper/1904.03751/) (2019) - [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016) - [SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos](https://mlsanity.com/paper/1804.04527/) (2018) - [SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos](https://mlsanity.com/paper/2011.13367/) (2020) - [Hierarchical Deep Temporal Models for Group Activity Recognition](https://mlsanity.com/paper/1607.02643/) (2016) - [Actor-Transformers for Group Activity Recognition](https://mlsanity.com/paper/2003.12737/) (2020) - [SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos](https://mlsanity.com/paper/2204.06918/) (2022) - [GroupFormer: Group Activity Recognition with Clustered Spatial-Temporal Transformer](https://mlsanity.com/paper/2108.12630/) (2021) - [Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition](https://mlsanity.com/paper/1611.09078/) (2016) - [Camera Calibration and Player Localization in SoccerNet-v2 and Investigation of their Representations for Action Spotting](https://mlsanity.com/paper/2104.09333/) (2021) - [Social Adaptive Module for Weakly-supervised Group Activity Recognition](https://mlsanity.com/paper/2007.09470/) (2020) - [SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimap](https://mlsanity.com/paper/2404.11335/) (2024) - [SoccerNet 2023 Challenges Results](https://mlsanity.com/paper/2309.06006/) (2023) - [COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality](https://mlsanity.com/paper/2112.05892/) (2021) - [SoccerNet-Echoes: A Soccer Game Audio Commentary Dataset](https://mlsanity.com/paper/2405.07354/) (2024) - [Towards More Practical Group Activity Detection: A New Benchmark and Model](https://mlsanity.com/paper/2312.02878/) (2023) - [A Graph-Based Method for Soccer Action Spotting Using Unsupervised Player Classification](https://mlsanity.com/paper/2211.12334/) (2022) - [Deep learning for action spotting in association football videos](https://mlsanity.com/paper/2410.01304/) (2024) - [A Graph Attention Based Approach for Trajectory Prediction in Multi-agent Sports Games](https://mlsanity.com/paper/2012.10531/) (2020) - [Skeleton-based Group Activity Recognition via Spatial-Temporal Panoramic Graph](https://mlsanity.com/paper/2407.19497/) (2024) ## Built on by (most cited first, 2 of 2) - [Towards Athlete Fatigue Assessment from Association Football Videos](https://mlsanity.com/paper/2604.05636/) (2026) - [SoccerLens: Grounded Soccer Video Understanding Beyond Accuracy](https://mlsanity.com/paper/2605.09598/) (2026) ## Abstract (arXiv) Group Activity Recognition (GAR) is well studied on the video modality for surveillance and indoor team sports (e.g., volleyball, basketball). Yet, other modalities such as agent positions and trajectories over time, i.e. tracking, remain comparatively under-explored despite being compact, agent-centric signals that explicitly encode spatial interactions. Understanding whether pixel (video) or position (tracking) modalities leads to better group activity recognition is therefore important to drive further research on the topic. However, no standardized benchmark currently exists that aligns broadcast video and tracking data for the same group activities, leading to a lack of apples-to-apples comparison between these modalities for GAR. In this work, we introduce SoccerNet-GAR, a multimodal dataset built from the 64 matches of the football World Cup 2022. Specifically, the broadcast videos and player tracking modalities for 87,939 group activities are synchronized and annotated with 10 categories. Furthermore, we define a unified evaluation protocol to benchmark two strong unimodal approaches: (i) competitive video-based classifiers and (ii) tracking-based classifiers leveraging graph neural networks. In particular, our novel role-aware graph architecture for tracking-based GAR directly encodes tactical structure through positional edges connecting players by their on-pitch roles. Our tracking model achieves 77.8% balanced accuracy compared to 60.9% for the best video baseline, while training with 7 × less GPU hours and 479 × fewer parameters (180K vs. 86.3M). This study provides new insights into the relative strengths of pixels and positions for group activity recognition in sports.