# Automatic Context-Driven Inference of Engagement in HMI: A Survey

arXiv: https://arxiv.org/abs/2209.15370 · HTML: https://mlsanity.com/paper/2209.15370/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 186 references that are in the map.
- First posted on arXiv 30 Sep 2022.

## Builds on (most cited first, 13 of 13)

- [Investigating Bias and Fairness in Facial Expression Recognition](https://mlsanity.com/paper/2007.10075/) (2020)
- [DAiSEE: Towards User Engagement Recognition in the Wild](https://mlsanity.com/paper/1609.01885/) (2016)
- [Prediction and Localization of Student Engagement in the Wild](https://mlsanity.com/paper/1804.00858/) (2018)
- [Personalized Machine Learning for Robot Perception of Affect and Engagement in Autism Therapy](https://mlsanity.com/paper/1802.01186/) (2018)
- [Engaging with Massive Online Courses](https://mlsanity.com/paper/1403.3100/) (2014)
- [Are you still with me? Continuous Engagement Assessment from a Robot's Point of View](https://mlsanity.com/paper/2001.03515/) (2020)
- [Multi-modal Active Learning From Human Data: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1906.03098/) (2019)
- [Towards engagement models that consider individual factors in HRI: on the relation of extroversion and negative attitude towards robots to gaze and speech during a human-robot assembly task](https://mlsanity.com/paper/1508.04603/) (2015)
- [A Deep Learning Approach for Multi-View Engagement Estimation of Children in a Child-Robot Joint Attention task](https://mlsanity.com/paper/1812.00253/) (2018)
- [On-the-fly Detection of User Engagement Decrease in Spontaneous Human-Robot Interaction, International Journal of Social Robotics, 2019](https://mlsanity.com/paper/2004.09596/) (2020)
- [Starting engagement detection towards a companion robot using multimodal features](https://mlsanity.com/paper/1503.03732/) (2015)
- [Engagement Detection in Meetings](https://mlsanity.com/paper/1608.08711/) (2016)
- [Vision-based Engagement Detection in Virtual Reality](https://mlsanity.com/paper/1609.01344/) (2016)

## Built on by (most cited first, 4 of 4)

- [Analyzing Participants' Engagement during Online Meetings Using Unsupervised Remote Photoplethysmography with Behavioral Features](https://mlsanity.com/paper/2404.04394/) (2024)
- [Computational Analysis of Stress, Depression and Engagement in Mental Health: A Survey](https://mlsanity.com/paper/2403.08824/) (2024)
- [Human-in-the-Loop Annotation for Image-Based Engagement Estimation: Assessing the Impact of Model Reliability on Annotation Accuracy](https://mlsanity.com/paper/2502.07404/) (2025)
- [SafeTalkCoach: Diversity-Driven Multi-Agent Simulation for Parent-Teen Health Conversations](https://mlsanity.com/paper/2602.00017/) (2026)

## Abstract (arXiv)

An integral part of seamless human-human communication is engagement, the process by which two or more participants establish, maintain, and end their perceived connection. Therefore, to develop successful human-centered human-machine interaction applications, automatic engagement inference is one of the tasks required to achieve engaging interactions between humans and machines, and to make machines attuned to their users, hence enhancing user satisfaction and technology acceptance. Several factors contribute to engagement state inference, which include the interaction context and interactants' behaviours and identity. Indeed, engagement is a multi-faceted and multi-modal construct that requires high accuracy in the analysis and interpretation of contextual, verbal and non-verbal cues. Thus, the development of an automated and intelligent system that accomplishes this task has been proven to be challenging so far. This paper presents a comprehensive survey on previous work in engagement inference for human-machine interaction, entailing interdisciplinary definition, engagement components and factors, publicly available datasets, ground truth assessment, and most commonly used features and methods, serving as a guide for the development of future human-machine interaction interfaces with reliable context-aware engagement inference capability. An in-depth review across embodied and disembodied interaction modes, and an emphasis on the interaction context of which engagement perception modules are integrated sets apart the presented survey from existing surveys.