# Visual Social Relationship Recognition

arXiv: https://arxiv.org/abs/1812.05917 · HTML: https://mlsanity.com/paper/1812.05917/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2018.

## Built on by (most cited first, 10 of 10)

- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [AUG: A New Dataset and An Efficient Model for Aerial Image Urban Scene Graph Generation](https://mlsanity.com/paper/2404.07788/) (2024)
- [Seeing the Intangible: Survey of Image Classification into High-Level and Abstract Categories](https://mlsanity.com/paper/2308.10562/) (2023)
- [Human-to-Human Interaction Detection](https://mlsanity.com/paper/2307.00464/) (2023)
- [XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detection](https://mlsanity.com/paper/2607.03562/) (2026)
- [Learning to Minimize the Remainder in Supervised Learning](https://mlsanity.com/paper/2201.09193/) (2022)
- [Humans Social Relationship Classification during Accompaniment](https://mlsanity.com/paper/2207.02890/) (2022)
- [On Support Relations Inference and Scene Hierarchy Graph Construction from Point Cloud in Clustered Environments](https://mlsanity.com/paper/2404.13842/) (2024)
- [From a Social Cognitive Perspective: Context-aware Visual Social Relationship Recognition](https://mlsanity.com/paper/2406.08358/) (2024)
- [Spot The Ball: A Benchmark for Visual Social Inference](https://mlsanity.com/paper/2511.00261/) (2025)

## Abstract (arXiv)

Social relationships form the basis of social structure of humans. Developing computational models to understand social relationships from visual data is essential for building intelligent machines that can better interact with humans in a social environment. In this work, we study the problem of visual social relationship recognition in images. We propose a Dual-Glance model for social relationship recognition, where the first glance fixates at the person of interest and the second glance deploys attention mechanism to exploit contextual cues. To enable this study, we curated a large scale People in Social Context (PISC) dataset, which comprises of 23,311 images and 79,244 person pairs with annotated social relationships. Since visually identifying social relationship bears certain degree of uncertainty, we further propose an Adaptive Focal Loss to leverage the ambiguous annotations for more effective learning. We conduct extensive experiments to quantitatively and qualitatively demonstrate the efficacy of our proposed method, which yields state-of-the-art performance on social relationship recognition.