# Prediction and Localization of Student Engagement in the Wild

arXiv: https://arxiv.org/abs/1804.00858 · HTML: https://mlsanity.com/paper/1804.00858/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2018.

## Built on by (most cited first, 30 of 33)

- [EduAgent: Generative Student Agents in Learning](https://mlsanity.com/paper/2404.07963/) (2024)
- [Automatic Gaze Analysis: A Survey of Deep Learning based Approaches](https://mlsanity.com/paper/2108.05479/) (2021)
- [Multimodal Engagement Analysis from Facial Videos in the Classroom](https://mlsanity.com/paper/2101.04215/) (2021)
- [Do I Have Your Attention: A Large Scale Engagement Prediction Dataset and Baselines](https://mlsanity.com/paper/2302.00431/) (2023)
- [Affect-driven Ordinal Engagement Measurement from Video](https://mlsanity.com/paper/2106.10882/) (2021)
- [MTGLS: Multi-Task Gaze Estimation with Limited Supervision](https://mlsanity.com/paper/2110.12100/) (2021)
- [DMCNet: Diversified Model Combination Network for Understanding Engagement from Video Screengrabs](https://mlsanity.com/paper/2204.06454/) (2022)
- [Class-attention Video Transformer for Engagement Intensity Prediction](https://mlsanity.com/paper/2208.07216/) (2022)
- [Detecting Disengagement in Virtual Learning as an Anomaly using Temporal Convolutional Network Autoencoder](https://mlsanity.com/paper/2211.06870/) (2022)
- [Bag of States: A Non-sequential Approach to Video-based Engagement Measurement](https://mlsanity.com/paper/2301.06730/) (2023)
- [Analyzing Participants' Engagement during Online Meetings Using Unsupervised Remote Photoplethysmography with Behavioral Features](https://mlsanity.com/paper/2404.04394/) (2024)
- [Automatic Context-Driven Inference of Engagement in HMI: A Survey](https://mlsanity.com/paper/2209.15370/) (2022)
- [Computational Analysis of Stress, Depression and Engagement in Mental Health: A Survey](https://mlsanity.com/paper/2403.08824/) (2024)
- [A General Model for Detecting Learner Engagement: Implementation and Evaluation](https://mlsanity.com/paper/2405.04251/) (2024)
- [DAT: Dialogue-Aware Transformer with Modality-Group Fusion for Human Engagement Estimation](https://mlsanity.com/paper/2410.08470/) (2024)
- [CMOSE: Comprehensive Multi-Modality Online Student Engagement Dataset with High-Quality Labels](https://mlsanity.com/paper/2312.09066/) (2023)
- [Can Population-based Engagement Improve Personalisation? A Novel Dataset and Experiments](https://mlsanity.com/paper/2207.01504/) (2022)
- [Supervised Contrastive Learning for Ordinal Engagement Measurement](https://mlsanity.com/paper/2505.20676/) (2025)
- [Context-Aware Academic Emotion Dataset and Benchmark](https://mlsanity.com/paper/2507.00586/) (2025)
- [Gems: Group Emotion Profiling Through Multimodal Situational Understanding](https://mlsanity.com/paper/2507.22393/) (2025)
- [PriorNet: Prior-Guided Engagement Estimation from Face Video](https://mlsanity.com/paper/2605.03615/) (2026)
- ['Labelling the Gaps': A Weakly Supervised Automatic Eye Gaze Estimation](https://mlsanity.com/paper/2208.01840/) (2022)
- [RAZE: Region Guided Self-Supervised Gaze Representation Learning](https://mlsanity.com/paper/2208.02485/) (2022)
- [Inconsistencies in the Definition and Annotation of Student Engagement in Virtual Learning Datasets: A Critical Review](https://mlsanity.com/paper/2208.04548/) (2022)
- [A survey of smart classroom: Concept, technologies and facial emotions recognition application](https://mlsanity.com/paper/2212.01675/) (2022)
- [Real-time estimation of overt attention from dynamic features of the face using deep-learning](https://mlsanity.com/paper/2409.13084/) (2024)
- [DIPSER: A Dataset for In-Person Student Engagement Recognition in the Wild](https://mlsanity.com/paper/2502.20209/) (2025)
- [United we stand, Divided we fall: Handling Weak Complementary Relationships for Audio-Visual Emotion Recognition in Valence-Arousal Space](https://mlsanity.com/paper/2503.12261/) (2025)
- [OPEN: A Benchmark Dataset and Baseline for Older Adult Patient Engagement Recognition in Virtual Rehabilitation Learning Environments](https://mlsanity.com/paper/2507.17959/) (2025)
- [Safeguarding Privacy: Privacy-Preserving Detection of Mind Wandering and Disengagement Using Federated Learning in Online Education](https://mlsanity.com/paper/2602.09904/) (2026)

## Abstract (arXiv)

In this paper, we introduce a new dataset for student engagement detection and localization. Digital revolution has transformed the traditional teaching procedure and a result analysis of the student engagement in an e-learning environment would facilitate effective task accomplishment and learning. Well known social cues of engagement/disengagement can be inferred from facial expressions, body movements and gaze pattern. In this paper, student's response to various stimuli videos are recorded and important cues are extracted to estimate variations in engagement level. In this paper, we study the association of a subject's behavioral cues with his/her engagement level, as annotated by labelers. We then localize engaging/non-engaging parts in the stimuli videos using a deep multiple instance learning based framework, which can give useful insight into designing Massive Open Online Courses (MOOCs) video material. Recognizing the lack of any publicly available dataset in the domain of user engagement, a new `in the wild' dataset is created to study the subject engagement problem. The dataset contains 195 videos captured from 78 subjects which is about 16.5 hours of recording. We present detailed baseline results using different classifiers ranging from traditional machine learning to deep learning based approaches. The subject independent analysis is performed so that it can be generalized to new users. The problem of engagement prediction is modeled as a weakly supervised learning problem. The dataset is manually annotated by different labelers for four levels of engagement independently and the correlation studies between annotated and predicted labels of videos by different classifiers is reported. This dataset creation is an effort to facilitate research in various e-learning environments such as intelligent tutoring systems, MOOCs, and others.