# Self-supervised Social Relation Representation for Human Group Detection

arXiv: https://arxiv.org/abs/2203.03843 · HTML: https://mlsanity.com/paper/2203.03843/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 50 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [Moments in Time Dataset: one million videos for event understanding](https://mlsanity.com/paper/1801.03150/) (2018)
- [End-to-End Human Object Interaction Detection with HOI Transformer](https://mlsanity.com/paper/2103.04503/) (2021)
- [Actor-Transformers for Group Activity Recognition](https://mlsanity.com/paper/2003.12737/) (2020)
- [Learning Actor Relation Graphs for Group Activity Recognition](https://mlsanity.com/paper/1904.10117/) (2019)
- [UniPose: Unified Human Pose Estimation in Single Images and Videos](https://mlsanity.com/paper/2001.08095/) (2020)
- [PANDA: A Gigapixel-level Human-centric Video Dataset](https://mlsanity.com/paper/2003.04852/) (2020)
- [Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition](https://mlsanity.com/paper/1611.09078/) (2016)
- [Social Adaptive Module for Weakly-supervised Group Activity Recognition](https://mlsanity.com/paper/2007.09470/) (2020)
- [Understanding Human Gaze Communication by Spatio-Temporal Graph Reasoning](https://mlsanity.com/paper/1909.02144/) (2019)
- [Cascaded Human-Object Interaction Recognition](https://mlsanity.com/paper/2003.04262/) (2020)
- [Joint Learning of Social Groups, Individuals Action and Sub-group Activities in Videos](https://mlsanity.com/paper/2007.02632/) (2020)
- [JRDB-Act: A Large-scale Dataset for Spatio-temporal Action, Social Group and Activity Detection](https://mlsanity.com/paper/2106.08827/) (2021)
- [Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition](https://mlsanity.com/paper/1809.01407/) (2018)
- [GD-GAN: Generative Adversarial Networks for Trajectory Prediction and Group Detection in Crowds](https://mlsanity.com/paper/1812.07667/) (2018)
- [Graph-Based Social Relation Reasoning](https://mlsanity.com/paper/2007.07453/) (2020)
- [An End-to-End Network for Generating Social Relationship Graphs](https://mlsanity.com/paper/1903.09784/) (2019)
- [Improving Social Awareness Through DANTE: A Deep Affinity Network for Clustering Conversational Interactants](https://mlsanity.com/paper/1907.12910/) (2019)

## Built on by (most cited first, 7 of 7)

- [Towards More Practical Group Activity Detection: A New Benchmark and Model](https://mlsanity.com/paper/2312.02878/) (2023)
- [GSON: A Group-based Social Navigation Framework with Large Multimodal Model](https://mlsanity.com/paper/2409.18084/) (2024)
- [Real-time Trajectory-based Social Group Detection](https://mlsanity.com/paper/2304.05678/) (2023)
- [Dynamic Group Detection using VLM-augmented Temporal Groupness Graph](https://mlsanity.com/paper/2509.04758/) (2025)
- [Social-MAE: Social Masked Autoencoder for Multi-person Motion Representation Learning](https://mlsanity.com/paper/2404.05578/) (2024)
- [End-to-End Shared Attention Estimation via Group Detection with Feedback Refinement](https://mlsanity.com/paper/2604.01714/) (2026)
- [Where Should I Join? Robot Group Joining via Language-Guided Goal Prediction](https://mlsanity.com/paper/2609.28467/) (2026)

## Abstract (arXiv)

Human group detection, which splits crowd of people into groups, is an important step for video-based human social activity analysis. The core of human group detection is the human social relation representation and division.In this paper, we propose a new two-stage multi-head framework for human group detection. In the first stage, we propose a human behavior simulator head to learn the social relation feature embedding, which is self-supervisely trained by leveraging the socially grounded multi-person behavior relationship. In the second stage, based on the social relation embedding, we develop a self-attention inspired network for human group detection. Remarkable performance on two state-of-the-art large-scale benchmarks, i.e., PANDA and JRDB-Group, verifies the effectiveness of the proposed framework. Benefiting from the self-supervised social relation embedding, our method can provide promising results with very few (labeled) training data. We will release the source code to the public.