# AI-based Multimodal Biometrics for Detecting Smartphone Distractions: Application to Online Learning

arXiv: https://arxiv.org/abs/2506.17364 · HTML: https://mlsanity.com/paper/2506.17364/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 53 references that are in the map.
- First posted on arXiv 20 Jun 2025.

## Builds on (most cited first, 18 of 18)

- [Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment](https://mlsanity.com/paper/2302.10908/) (2023)
- [TypeNet: Deep Learning Keystroke Biometrics](https://mlsanity.com/paper/2101.05570/) (2021)
- [edBB-Demo: Biometrics and Behavior Analysis for Online Educational Platforms](https://mlsanity.com/paper/2211.09210/) (2022)
- [M2LADS: A System for Generating MultiModal Learning Analytics Dashboards in Open Education](https://mlsanity.com/paper/2305.12561/) (2023)
- [ALEBk: Feasibility Study of Attention Level Estimation via Blink Detection applied to e-Learning](https://mlsanity.com/paper/2112.09165/) (2021)
- [BeCAPTCHA-Mouse: Synthetic Mouse Trajectories and Improved Bot Detection](https://mlsanity.com/paper/2005.00890/) (2020)
- [VAAD: Visual Attention Analysis Dashboard applied to e-Learning](https://mlsanity.com/paper/2405.20091/) (2024)
- [A multimodal dataset for understanding the impact of mobile phones on remote online virtual education](https://mlsanity.com/paper/2412.14195/) (2024)
- [Biometrics and Behavior Analysis for Detecting Distractions in e-Learning](https://mlsanity.com/paper/2405.15434/) (2024)
- [mEBAL2 Database and Benchmark: Image-based Multispectral Eyeblink Detection](https://mlsanity.com/paper/2309.07880/) (2023)
- [DeepFace-Attention: Multimodal Face Biometrics for Attention Estimation with Application to e-Learning](https://mlsanity.com/paper/2408.05523/) (2024)
- [A Comparative Evaluation of Heart Rate Estimation Methods using Face Videos](https://mlsanity.com/paper/2005.11101/) (2020)
- [SMARTe-VR: Student Monitoring and Adaptive Response Technology for e-Learning in Virtual Reality](https://mlsanity.com/paper/2501.10977/) (2025)
- [MOSAIC-F: A Framework for Enhancing Students' Oral Presentation Skills through Personalized Feedback](https://mlsanity.com/paper/2506.08634/) (2025)
- [Quality-based Pulse Estimation from NIR Face Video with Application to Driver Monitoring](https://mlsanity.com/paper/1905.06568/) (2019)
- [KVC-onGoing: Keystroke Verification Challenge](https://mlsanity.com/paper/2412.20530/) (2024)
- [M2LADS Demo: A System for Generating Multimodal Learning Analytics Dashboards](https://mlsanity.com/paper/2502.15363/) (2025)
- [From Complexity to Parsimony: Integrating Latent Class Analysis to Uncover Multimodal Learning Patterns in Collaborative Learning](https://mlsanity.com/paper/2411.15590/) (2024)

## Built on by (most cited first, 6 of 6)

- [PBa-LLM: Privacy- and Bias-aware NLP using Named-Entity Recognition (NER)](https://mlsanity.com/paper/2507.02966/) (2025)
- [EduEVAL-DB: A Role-Based Dataset for Pedagogical Risk Evaluation in Educational Explanations](https://mlsanity.com/paper/2602.15531/) (2026)
- [Enhancing Online Learning by Integrating Biosensors and Multimodal Learning Analytics for Detecting and Predicting Student Behavior: A Review](https://mlsanity.com/paper/2509.07742/) (2025)
- [AIriskEval-edu: New Dataset for Risk Assessment in AI-mediated K-12 Educational Explanations](https://mlsanity.com/paper/2607.01934/) (2026)
- [A Multimodal Dataset of Student Oral Presentations with Sensors and Evaluation Data](https://mlsanity.com/paper/2601.07576/) (2026)
- [Real-Time Multimodal Data Collection Using Smartwatches and Its Visualization in Education](https://mlsanity.com/paper/2512.02651/) (2025)

## Abstract (arXiv)

This work investigates the use of multimodal biometrics to detect distractions caused by smartphone use during tasks that require sustained attention, with a focus on computer-based online learning. Although the methods are applicable to various domains, such as autonomous driving, we concentrate on the challenges learners face in maintaining engagement amid internal (e.g., motivation), system-related (e.g., course design) and contextual (e.g., smartphone use) factors. Traditional learning platforms often lack detailed behavioral data, but Multimodal Learning Analytics (MMLA) and biosensors provide new insights into learner attention. We propose an AI-based approach that leverages physiological signals and head pose data to detect phone use. Our results show that single biometric signals, such as brain waves or heart rate, offer limited accuracy, while head pose alone achieves 87%. A multimodal model combining all signals reaches 91% accuracy, highlighting the benefits of integration. We conclude by discussing the implications and limitations of deploying these models for real-time support in online learning environments.