# An End-to-End Network for Generating Social Relationship Graphs

arXiv: https://arxiv.org/abs/1903.09784 · HTML: https://mlsanity.com/paper/1903.09784/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Mar 2019.

## Built on by (most cited first, 11 of 11)

- [Self-supervised Social Relation Representation for Human Group Detection](https://mlsanity.com/paper/2203.03843/) (2022)
- [Query Driven-Graph Neural Networks for Community Search: From Non-Attributed, Attributed, to Interactive Attributed](https://mlsanity.com/paper/2104.03583/) (2021)
- [Pairwise Emotional Relationship Recognition in Drama Videos: Dataset and Benchmark](https://mlsanity.com/paper/2109.11243/) (2021)
- [Graph Neural Networks in Computer Vision -- Architectures, Datasets and Common Approaches](https://mlsanity.com/paper/2212.10207/) (2022)
- [Seeing the Intangible: Survey of Image Classification into High-Level and Abstract Categories](https://mlsanity.com/paper/2308.10562/) (2023)
- [Stitching Gaps: Fusing Situated Perceptual Knowledge with Vision Transformers for High-Level Image Classification](https://mlsanity.com/paper/2402.19339/) (2024)
- [SocialFusion: Addressing Social Degradation in Pre-trained Vision-Language Models](https://mlsanity.com/paper/2512.01148/) (2025)
- [Enhancing Social Relation Inference with Concise Interaction Graph and Discriminative Scene Representation](https://mlsanity.com/paper/2107.14425/) (2021)
- [GRITv2: Efficient and Light-weight Social Relation Recognition](https://mlsanity.com/paper/2403.06895/) (2024)
- [From a Social Cognitive Perspective: Context-aware Visual Social Relationship Recognition](https://mlsanity.com/paper/2406.08358/) (2024)
- [CloSE: A Geometric Shape-Agnostic Cloth State Representation](https://mlsanity.com/paper/2504.05033/) (2025)

## Abstract (arXiv)

Socially-intelligent agents are of growing interest in artificial intelligence. To this end, we need systems that can understand social relationships in diverse social contexts. Inferring the social context in a given visual scene not only involves recognizing objects, but also demands a more in-depth understanding of the relationships and attributes of the people involved. To achieve this, one computational approach for representing human relationships and attributes is to use an explicit knowledge graph, which allows for high-level reasoning. We introduce a novel end-to-end-trainable neural network that is capable of generating a Social Relationship Graph - a structured, unified representation of social relationships and attributes - from a given input image. Our Social Relationship Graph Generation Network (SRG-GN) is the first to use memory cells like Gated Recurrent Units (GRUs) to iteratively update the social relationship states in a graph using scene and attribute context. The neural network exploits the recurrent connections among the GRUs to implement message passing between nodes and edges in the graph, and results in significant improvement over previous methods for social relationship recognition.