# MATT: Multimodal Attention Level Estimation for e-learning Platforms

arXiv: https://arxiv.org/abs/2301.09174 · HTML: https://mlsanity.com/paper/2301.09174/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 44 references that are in the map.
- First posted on arXiv 22 Jan 2023.

## Builds on (most cited first, 7 of 7)

- [WIDER FACE: A Face Detection Benchmark](https://mlsanity.com/paper/1511.06523/) (2015)
- [Style Aggregated Network for Facial Landmark Detection](https://mlsanity.com/paper/1803.04108/) (2018)
- [mEBAL: A Multimodal Database for Eye Blink Detection and Attention Level Estimation](https://mlsanity.com/paper/2006.05327/) (2020)
- [edBB: Biometrics and Behavior for Assessing Remote Education](https://mlsanity.com/paper/1912.04786/) (2019)
- [ALEBk: Feasibility Study of Attention Level Estimation via Blink Detection applied to e-Learning](https://mlsanity.com/paper/2112.09165/) (2021)
- [Heart Rate Estimation from Face Videos for Student Assessment: Experiments on edBB](https://mlsanity.com/paper/2006.00825/) (2020)
- [RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild](https://mlsanity.com/paper/2011.01890/) (2020)

## Built on by (most cited first, 7 of 7)

- [M2LADS: A System for Generating MultiModal Learning Analytics Dashboards in Open Education](https://mlsanity.com/paper/2305.12561/) (2023)
- [Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Data](https://mlsanity.com/paper/2412.01383/) (2024)
- [VAAD: Visual Attention Analysis Dashboard applied to e-Learning](https://mlsanity.com/paper/2405.20091/) (2024)
- [Biometrics and Behavior Analysis for Detecting Distractions in e-Learning](https://mlsanity.com/paper/2405.15434/) (2024)
- [mEBAL2 Database and Benchmark: Image-based Multispectral Eyeblink Detection](https://mlsanity.com/paper/2309.07880/) (2023)
- [SMARTe-VR: Student Monitoring and Adaptive Response Technology for e-Learning in Virtual Reality](https://mlsanity.com/paper/2501.10977/) (2025)
- [M&M: Multimodal-Multitask Model Integrating Audiovisual Cues in Cognitive Load Assessment](https://mlsanity.com/paper/2403.09451/) (2024)

## Abstract (arXiv)

This work presents a new multimodal system for remote attention level estimation based on multimodal face analysis. Our multimodal approach uses different parameters and signals obtained from the behavior and physiological processes that have been related to modeling cognitive load such as faces gestures (e.g., blink rate, facial actions units) and user actions (e.g., head pose, distance to the camera). The multimodal system uses the following modules based on Convolutional Neural Networks (CNNs): Eye blink detection, head pose estimation, facial landmark detection, and facial expression features. First, we individually evaluate the proposed modules in the task of estimating the student's attention level captured during online e-learning sessions. For that we trained binary classifiers (high or low attention) based on Support Vector Machines (SVM) for each module. Secondly, we find out to what extent multimodal score level fusion improves the attention level estimation. The mEBAL database is used in the experimental framework, a public multi-modal database for attention level estimation obtained in an e-learning environment that contains data from 38 users while conducting several e-learning tasks of variable difficulty (creating changes in student cognitive loads).