# Learning Social Relation Traits from Face Images

arXiv: https://arxiv.org/abs/1509.03936 · HTML: https://mlsanity.com/paper/1509.03936/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2015.

## Built on by (most cited first, 25 of 25)

- [Understanding and Mitigating Annotation Bias in Facial Expression Recognition](https://mlsanity.com/paper/2108.08504/) (2021)
- [SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network](https://mlsanity.com/paper/2110.12334/) (2021)
- [ExpLLM: Towards Chain of Thought for Facial Expression Recognition](https://mlsanity.com/paper/2409.02828/) (2024)
- [Did the Model Change? Efficiently Assessing Machine Learning API Shifts](https://mlsanity.com/paper/2107.14203/) (2021)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [Facial Expression Recognition based on Multi-head Cross Attention Network](https://mlsanity.com/paper/2203.13235/) (2022)
- [DMCNet: Diversified Model Combination Network for Understanding Engagement from Video Screengrabs](https://mlsanity.com/paper/2204.06454/) (2022)
- [Affective Behaviour Analysis via Progressive Learning](https://mlsanity.com/paper/2407.16945/) (2024)
- [HAPI: A Large-scale Longitudinal Dataset of Commercial ML API Predictions](https://mlsanity.com/paper/2209.08443/) (2022)
- [Learning from Synthetic Data: Facial Expression Classification based on Ensemble of Multi-task Networks](https://mlsanity.com/paper/2207.10025/) (2022)
- [Seeking Subjectivity in Visual Emotion Distribution Learning](https://mlsanity.com/paper/2207.11875/) (2022)
- [Meta-Mining Discriminative Samples for Kinship Verification](https://mlsanity.com/paper/2103.15108/) (2021)
- [Seeing the Intangible: Survey of Image Classification into High-Level and Abstract Categories](https://mlsanity.com/paper/2308.10562/) (2023)
- [Regional Attention Network (RAN) for Head Pose and Fine-grained Gesture Recognition](https://mlsanity.com/paper/2101.06634/) (2021)
- [Multi LoRA Meets Vision: Merging multiple adapters to create a multi task model](https://mlsanity.com/paper/2411.14064/) (2024)
- [Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding](https://mlsanity.com/paper/2508.20765/) (2025)
- [Mitigating Bias in Facial Analysis Systems by Incorporating Label Diversity](https://mlsanity.com/paper/2204.06364/) (2022)
- [Improving Identity-Robustness for Face Models](https://mlsanity.com/paper/2304.03838/) (2023)
- [Early Joint Learning of Emotion Information Makes MultiModal Model Understand You Better](https://mlsanity.com/paper/2409.18971/) (2024)
- [Bayesian Convolutional Neural Networks for Seven Basic Facial Expression Classifications](https://mlsanity.com/paper/2107.04834/) (2021)
- [Enhancing Social Relation Inference with Concise Interaction Graph and Discriminative Scene Representation](https://mlsanity.com/paper/2107.14425/) (2021)
- [Evaluation of Self-taught Learning-based Representations for Facial Emotion Recognition](https://mlsanity.com/paper/2204.12624/) (2022)
- [Exploring Interactions and Regulations in Collaborative Learning: An Interdisciplinary Multimodal Dataset](https://mlsanity.com/paper/2210.05419/) (2022)
- [From a Social Cognitive Perspective: Context-aware Visual Social Relationship Recognition](https://mlsanity.com/paper/2406.08358/) (2024)
- [A survey on Graph Deep Representation Learning for Facial Expression Recognition](https://mlsanity.com/paper/2411.08472/) (2024)

## Abstract (arXiv)

Social relation defines the association, e.g, warm, friendliness, and dominance, between two or more people. Motivated by psychological studies, we investigate if such fine-grained and high-level relation traits can be characterised and quantified from face images in the wild. To address this challenging problem we propose a deep model that learns a rich face representation to capture gender, expression, head pose, and age-related attributes, and then performs pairwise-face reasoning for relation prediction. To learn from heterogeneous attribute sources, we formulate a new network architecture with a bridging layer to leverage the inherent correspondences among these datasets. It can also cope with missing target attribute labels. Extensive experiments show that our approach is effective for fine-grained social relation learning in images and videos.