# Human-Inspired Social Engagement Analysis via Interpretable Mutual Visual Attention

arXiv: https://arxiv.org/abs/2608.24580 · HTML: https://mlsanity.com/paper/2608.24580/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 38 references that are in the map.
- First posted on arXiv 25 Aug 2026.

## Builds on (most cited first, 10 of 10)

- [Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances](https://mlsanity.com/paper/1905.02947/) (2019)
- [Towards Social Artificial Intelligence: Nonverbal Social Signal Prediction in A Triadic Interaction](https://mlsanity.com/paper/1906.04158/) (2019)
- [F-formation Detection: Individuating Free-standing Conversational Groups in Images](https://mlsanity.com/paper/1409.2702/) (2014)
- [HHP-Net: A light Heteroscedastic neural network for Head Pose estimation with uncertainty](https://mlsanity.com/paper/2111.01440/) (2021)
- [Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey](https://mlsanity.com/paper/2207.10574/) (2022)
- [MGTR: End-to-End Mutual Gaze Detection with Transformer](https://mlsanity.com/paper/2209.10930/) (2022)
- [Learning in Focus: Detecting Behavioral and Collaborative Engagement Using Vision Transformers](https://mlsanity.com/paper/2508.15782/) (2025)
- [Social Visual Behavior Analytics for Autism Therapy of Children Based on Automated Mutual Gaze Detection](https://mlsanity.com/paper/2302.08293/) (2023)
- [ViBED-Net: Video Based Engagement Detection Network Using Face-Aware and Scene-Aware Spatiotemporal Cues](https://mlsanity.com/paper/2510.18016/) (2025)
- [Modeling social interaction dynamics using temporal graph networks](https://mlsanity.com/paper/2404.06611/) (2024)

## Abstract (arXiv)

Understanding social interactions from non-verbal visual data is important for behavior analysis and activity monitoring. We propose an interpretable computational model of social engagement inspired by psychological theories of mutual visual attention. Rather than learning interaction patterns end-to-end, our framework explicitly models dyadic visual attention and aggregates these cues into interpretable measures of individual and group engagement. The resulting modular framework combines state-of-the-art head orientation estimation with lightweight geometric reasoning, producing explanations that remain accessible to non-technical users. We evaluate the proposed approach on a variety of data through quantitative experiments and demonstrate its practical usefulness with qualitative visualizations designed to support teachers, caregivers, and social workers in understanding group interaction dynamics.