# Deep Reasoning with Knowledge Graph for Social Relationship Understanding

arXiv: https://arxiv.org/abs/1807.00504 · HTML: https://mlsanity.com/paper/1807.00504/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Jul 2018.

## Built on by (most cited first, 30 of 44)

- [Graph Learning: A Survey](https://mlsanity.com/paper/2105.00696/) (2021)
- [Knowledge Graphs: Opportunities and Challenges](https://mlsanity.com/paper/2303.13948/) (2023)
- [Spatial Dual-Modality Graph Reasoning for Key Information Extraction](https://mlsanity.com/paper/2103.14470/) (2021)
- [PaSca: a Graph Neural Architecture Search System under the Scalable Paradigm](https://mlsanity.com/paper/2203.00638/) (2022)
- [Contrastive Language-Image Pre-Training with Knowledge Graphs](https://mlsanity.com/paper/2210.08901/) (2022)
- [Graph Lifelong Learning: A Survey](https://mlsanity.com/paper/2202.10688/) (2022)
- [threaTrace: Detecting and Tracing Host-based Threats in Node Level Through Provenance Graph Learning](https://mlsanity.com/paper/2111.04333/) (2021)
- [Dynamic Interactive Relation Capturing via Scene Graph Learning for Robotic Surgical Report Generation](https://mlsanity.com/paper/2306.02651/) (2023)
- [Graph Classification via Reference Distribution Learning: Theory and Practice](https://mlsanity.com/paper/2408.11370/) (2024)
- [OpenGU: A Comprehensive Benchmark for Graph Unlearning](https://mlsanity.com/paper/2501.02728/) (2025)
- [Survey of Image Based Graph Neural Networks](https://mlsanity.com/paper/2106.06307/) (2021)
- [Two-stage Rule-induction Visual Reasoning on RPMs with an Application to Video Prediction](https://mlsanity.com/paper/2111.12301/) (2021)
- [MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection](https://mlsanity.com/paper/2204.01349/) (2022)
- [Modeling the Trade-off of Privacy Preservation and Activity Recognition on Low-Resolution Images](https://mlsanity.com/paper/2303.10435/) (2023)
- [NeutronStream: A Dynamic GNN Training Framework with Sliding Window for Graph Streams](https://mlsanity.com/paper/2312.02473/) (2023)
- [GMLP: Building Scalable and Flexible Graph Neural Networks with Feature-Message Passing](https://mlsanity.com/paper/2104.09880/) (2021)
- [DRAG: Dynamic Region-Aware GCN for Privacy-Leaking Image Detection](https://mlsanity.com/paper/2203.09121/) (2022)
- [Content-based Graph Privacy Advisor](https://mlsanity.com/paper/2210.11169/) (2022)
- [Bregman Graph Neural Network](https://mlsanity.com/paper/2309.06645/) (2023)
- [GTAE: Graph-Transformer based Auto-Encoders for Linguistic-Constrained Text Style Transfer](https://mlsanity.com/paper/2102.00769/) (2021)
- [Human-to-Human Interaction Detection](https://mlsanity.com/paper/2307.00464/) (2023)
- [Explaining models relating objects and privacy](https://mlsanity.com/paper/2405.01646/) (2024)
- [Str-GCL: Structural Commonsense Driven Graph Contrastive Learning](https://mlsanity.com/paper/2507.07141/) (2025)
- [DGI: Easy and Efficient Inference for GNNs](https://mlsanity.com/paper/2211.15082/) (2022)
- [Spatio-Temporal driven Attention Graph Neural Network with Block Adjacency matrix (STAG-NN-BA) for Remote Land-use Change Detection](https://mlsanity.com/paper/2303.14322/) (2023)
- [Building A Knowledge Graph to Enrich ChatGPT Responses in Manufacturing Service Discovery](https://mlsanity.com/paper/2404.06571/) (2024)
- [D3-GNN: Dynamic Distributed Dataflow for Streaming Graph Neural Networks](https://mlsanity.com/paper/2409.09079/) (2024)
- [Learning Privacy from Visual Entities](https://mlsanity.com/paper/2503.12464/) (2025)
- [Bridging Dual Knowledge Graphs for Multi-Hop Question Answering in Construction Safety](https://mlsanity.com/paper/2507.13625/) (2025)
- [Learning Personalized Representations using Graph Convolutional Network](https://mlsanity.com/paper/2207.14298/) (2022)

## Abstract (arXiv)

Social relationships (e.g., friends, couple etc.) form the basis of the social network in our daily life. Automatically interpreting such relationships bears a great potential for the intelligent systems to understand human behavior in depth and to better interact with people at a social level. Human beings interpret the social relationships within a group not only based on the people alone, and the interplay between such social relationships and the contextual information around the people also plays a significant role. However, these additional cues are largely overlooked by the previous studies. We found that the interplay between these two factors can be effectively modeled by a novel structured knowledge graph with proper message propagation and attention. And this structured knowledge can be efficiently integrated into the deep neural network architecture to promote social relationship understanding by an end-to-end trainable Graph Reasoning Model (GRM), in which a propagation mechanism is learned to propagate node message through the graph to explore the interaction between persons of interest and the contextual objects. Meanwhile, a graph attentional mechanism is introduced to explicitly reason about the discriminative objects to promote recognition. Extensive experiments on the public benchmarks demonstrate the superiority of our method over the existing leading competitors.