# DeepFace-Attention: Multimodal Face Biometrics for Attention Estimation with Application to e-Learning

arXiv: https://arxiv.org/abs/2408.05523 · HTML: https://mlsanity.com/paper/2408.05523/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 79 references that are in the map.
- First posted on arXiv 10 Aug 2024.

## Builds on (most cited first, 19 of 19)

- [WIDER FACE: A Face Detection Benchmark](https://mlsanity.com/paper/1511.06523/) (2015)
- [DeepPhys: Video-Based Physiological Measurement Using Convolutional Attention Networks](https://mlsanity.com/paper/1805.07888/) (2018)
- [Style Aggregated Network for Facial Landmark Detection](https://mlsanity.com/paper/1803.04108/) (2018)
- [A Reproducible Study on Remote Heart Rate Measurement](https://mlsanity.com/paper/1709.00962/) (2017)
- [A Compact Embedding for Facial Expression Similarity](https://mlsanity.com/paper/1811.11283/) (2018)
- [mEBAL: A Multimodal Database for Eye Blink Detection and Attention Level Estimation](https://mlsanity.com/paper/2006.05327/) (2020)
- [Decoupled-and-Coupled Networks: Self-Supervised Hyperspectral Image Super-Resolution with Subpixel Fusion](https://mlsanity.com/paper/2205.03742/) (2022)
- [edBB-Demo: Biometrics and Behavior Analysis for Online Educational Platforms](https://mlsanity.com/paper/2211.09210/) (2022)
- [M2LADS: A System for Generating MultiModal Learning Analytics Dashboards in Open Education](https://mlsanity.com/paper/2305.12561/) (2023)
- [ALEBk: Feasibility Study of Attention Level Estimation via Blink Detection applied to e-Learning](https://mlsanity.com/paper/2112.09165/) (2021)
- [Heart Rate Estimation from Face Videos for Student Assessment: Experiments on edBB](https://mlsanity.com/paper/2006.00825/) (2020)
- [VAAD: Visual Attention Analysis Dashboard applied to e-Learning](https://mlsanity.com/paper/2405.20091/) (2024)
- [Biometrics and Behavior Analysis for Detecting Distractions in e-Learning](https://mlsanity.com/paper/2405.15434/) (2024)
- [mEBAL2 Database and Benchmark: Image-based Multispectral Eyeblink Detection](https://mlsanity.com/paper/2309.07880/) (2023)
- [RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild](https://mlsanity.com/paper/2011.01890/) (2020)
- [A Comparative Evaluation of Heart Rate Estimation Methods using Face Videos](https://mlsanity.com/paper/2005.11101/) (2020)
- [Keystroke Verification Challenge (KVC): Biometric and Fairness Benchmark Evaluation](https://mlsanity.com/paper/2311.06000/) (2023)
- [Quality-based Pulse Estimation from NIR Face Video with Application to Driver Monitoring](https://mlsanity.com/paper/1905.06568/) (2019)
- [Fusing CNNs and statistical indicators to improve image classification](https://mlsanity.com/paper/2012.11049/) (2020)

## Built on by (most cited first, 7 of 7)

- [A multimodal dataset for understanding the impact of mobile phones on remote online virtual education](https://mlsanity.com/paper/2412.14195/) (2024)
- [SMARTe-VR: Student Monitoring and Adaptive Response Technology for e-Learning in Virtual Reality](https://mlsanity.com/paper/2501.10977/) (2025)
- [AI-based Multimodal Biometrics for Detecting Smartphone Distractions: Application to Online Learning](https://mlsanity.com/paper/2506.17364/) (2025)
- [Exploiting Multiple Representations: 3D Face Biometrics Fusion with Application to Surveillance](https://mlsanity.com/paper/2504.18886/) (2025)
- [AttZoom: Attention Zoom for Better Visual Features](https://mlsanity.com/paper/2508.03625/) (2025)
- [EduEVAL-DB: A Role-Based Dataset for Pedagogical Risk Evaluation in Educational Explanations](https://mlsanity.com/paper/2602.15531/) (2026)
- [M2LADS Demo: A System for Generating Multimodal Learning Analytics Dashboards](https://mlsanity.com/paper/2502.15363/) (2025)

## Abstract (arXiv)

This work introduces an innovative method for estimating attention levels (cognitive load) using an ensemble of facial analysis techniques applied to webcam videos. Our method is particularly useful, among others, in e-learning applications, so we trained, evaluated, and compared our approach on the mEBAL2 database, a public multi-modal database acquired in an e-learning environment. mEBAL2 comprises data from 60 users who performed 8 different tasks. These tasks varied in difficulty, leading to changes in their cognitive loads. Our approach adapts state-of-the-art facial analysis technologies to quantify the users' cognitive load in the form of high or low attention. Several behavioral signals and physiological processes related to the cognitive load are used, such as eyeblink, heart rate, facial action units, and head pose, among others. Furthermore, we conduct a study to understand which individual features obtain better results, the most efficient combinations, explore local and global features, and how temporary time intervals affect attention level estimation, among other aspects. We find that global facial features are more appropriate for multimodal systems using score-level fusion, particularly as the temporal window increases. On the other hand, local features are more suitable for fusion through neural network training with score-level fusion approaches. Our method outperforms existing state-of-the-art accuracies using the public mEBAL2 benchmark.