# CERN: Confidence-Energy Recurrent Network for Group Activity Recognition

arXiv: https://arxiv.org/abs/1704.03058 · HTML: https://mlsanity.com/paper/1704.03058/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2017.

## Built on by (most cited first, 24 of 24)

- [GroupFormer: Group Activity Recognition with Clustered Spatial-Temporal Transformer](https://mlsanity.com/paper/2108.12630/) (2021)
- [Spatio-Temporal Dynamic Inference Network for Group Activity Recognition](https://mlsanity.com/paper/2108.11743/) (2021)
- [A Survey on Video Action Recognition in Sports: Datasets, Methods and Applications](https://mlsanity.com/paper/2206.01038/) (2022)
- [Dual-AI: Dual-path Actor Interaction Learning for Group Activity Recognition](https://mlsanity.com/paper/2204.02148/) (2022)
- [Hunting Group Clues with Transformers for Social Group Activity Recognition](https://mlsanity.com/paper/2207.05254/) (2022)
- [Detector-Free Weakly Supervised Group Activity Recognition](https://mlsanity.com/paper/2204.02139/) (2022)
- [COMPOSER: Compositional Reasoning of Group Activity in Videos with Keypoint-Only Modality](https://mlsanity.com/paper/2112.05892/) (2021)
- [Pose is all you need: The pose only group activity recognition system (POGARS)](https://mlsanity.com/paper/2108.04186/) (2021)
- [Panoramic Human Activity Recognition](https://mlsanity.com/paper/2203.03806/) (2022)
- [Towards More Practical Group Activity Detection: A New Benchmark and Model](https://mlsanity.com/paper/2312.02878/) (2023)
- [Skeleton-based Group Activity Recognition via Spatial-Temporal Panoramic Graph](https://mlsanity.com/paper/2407.19497/) (2024)
- [Learning Group Activities from Skeletons without Individual Action Labels](https://mlsanity.com/paper/2105.06754/) (2021)
- [Actor-centered Representations for Action Localization in Streaming Videos](https://mlsanity.com/paper/2104.14131/) (2021)
- [Human-to-Human Interaction Detection](https://mlsanity.com/paper/2307.00464/) (2023)
- [Knowledge Augmented Relation Inference for Group Activity Recognition](https://mlsanity.com/paper/2302.14350/) (2023)
- [DECOMPL: Decompositional Learning with Attention Pooling for Group Activity Recognition from a Single Volleyball Image](https://mlsanity.com/paper/2303.06439/) (2023)
- [Self-supervised Multi-actor Social Activity Understanding in Streaming Videos](https://mlsanity.com/paper/2406.14472/) (2024)
- [Group Activity Recognition using Unreliable Tracked Pose](https://mlsanity.com/paper/2401.03262/) (2024)
- [Design and Analysis of Efficient Attention in Transformers for Social Group Activity Recognition](https://mlsanity.com/paper/2404.09964/) (2024)
- [Unified Framework with Consistency across Modalities for Human Activity Recognition](https://mlsanity.com/paper/2409.02385/) (2024)
- [Attentive pooling for Group Activity Recognition](https://mlsanity.com/paper/2208.14847/) (2022)
- [Group Activity Recognition via Dynamic Composition and Interaction](https://mlsanity.com/paper/2305.05583/) (2023)
- [Group Activity Recognition in Computer Vision: A Comprehensive Review, Challenges, and Future Perspectives](https://mlsanity.com/paper/2307.13541/) (2023)
- [VicKAM: Visual Conceptual Knowledge Guided Action Map for Weakly Supervised Group Activity Recognition](https://mlsanity.com/paper/2502.09967/) (2025)

## Abstract (arXiv)

This work is about recognizing human activities occurring in videos at distinct semantic levels, including individual actions, interactions, and group activities. The recognition is realized using a two-level hierarchy of Long Short-Term Memory (LSTM) networks, forming a feed-forward deep architecture, which can be trained end-to-end. In comparison with existing architectures of LSTMs, we make two key contributions giving the name to our approach as Confidence-Energy Recurrent Network -- CERN. First, instead of using the common softmax layer for prediction, we specify a novel energy layer (EL) for estimating the energy of our predictions. Second, rather than finding the common minimum-energy class assignment, which may be numerically unstable under uncertainty, we specify that the EL additionally computes the p-values of the solutions, and in this way estimates the most confident energy minimum. The evaluation on the Collective Activity and Volleyball datasets demonstrates: (i) advantages of our two contributions relative to the common softmax and energy-minimization formulations and (ii) a superior performance relative to the state-of-the-art approaches.