# mEBAL2 Database and Benchmark: Image-based Multispectral Eyeblink Detection

arXiv: https://arxiv.org/abs/2309.07880 · HTML: https://mlsanity.com/paper/2309.07880/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 46 references that are in the map.
- First posted on arXiv 14 Sep 2023.

## Builds on (most cited first, 18 of 18)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Siamese Network for RGB-D Salient Object Detection and Beyond](https://mlsanity.com/paper/2008.12134/) (2020)
- [Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors](https://mlsanity.com/paper/1807.00966/) (2018)
- [Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment](https://mlsanity.com/paper/2302.10908/) (2023)
- [mEBAL: A Multimodal Database for Eye Blink Detection and Attention Level Estimation](https://mlsanity.com/paper/2006.05327/) (2020)
- [edBB-Demo: Biometrics and Behavior Analysis for Online Educational Platforms](https://mlsanity.com/paper/2211.09210/) (2022)
- [edBB: Biometrics and Behavior for Assessing Remote Education](https://mlsanity.com/paper/1912.04786/) (2019)
- [M2LADS: A System for Generating MultiModal Learning Analytics Dashboards in Open Education](https://mlsanity.com/paper/2305.12561/) (2023)
- [ALEBk: Feasibility Study of Attention Level Estimation via Blink Detection applied to e-Learning](https://mlsanity.com/paper/2112.09165/) (2021)
- [Introduction to Presentation Attack Detection in Face Biometrics and Recent Advances](https://mlsanity.com/paper/2111.11794/) (2021)
- [Periocular Biometrics: A Modality for Unconstrained Scenarios](https://mlsanity.com/paper/2212.13792/) (2022)
- [Cross-Sensor Periocular Biometrics in a Global Pandemic: Comparative Benchmark and Novel Multialgorithmic Approach](https://mlsanity.com/paper/1902.08123/) (2019)
- [MATT: Multimodal Attention Level Estimation for e-learning Platforms](https://mlsanity.com/paper/2301.09174/) (2023)
- [Towards Real-time Eyeblink Detection in The Wild:Dataset,Theory and Practices](https://mlsanity.com/paper/1902.07891/) (2019)
- [Real-time Multi-person Eyeblink Detection in the Wild for Untrimmed Video](https://mlsanity.com/paper/2303.16053/) (2023)

## Built on by (most cited first, 9 of 9)

- [VAAD: Visual Attention Analysis Dashboard applied to e-Learning](https://mlsanity.com/paper/2405.20091/) (2024)
- [A multimodal dataset for understanding the impact of mobile phones on remote online virtual education](https://mlsanity.com/paper/2412.14195/) (2024)
- [DeepFace-Attention: Multimodal Face Biometrics for Attention Estimation with Application to e-Learning](https://mlsanity.com/paper/2408.05523/) (2024)
- [SMARTe-VR: Student Monitoring and Adaptive Response Technology for e-Learning in Virtual Reality](https://mlsanity.com/paper/2501.10977/) (2025)
- [AI-based Multimodal Biometrics for Detecting Smartphone Distractions: Application to Online Learning](https://mlsanity.com/paper/2506.17364/) (2025)
- [EduEVAL-DB: A Role-Based Dataset for Pedagogical Risk Evaluation in Educational Explanations](https://mlsanity.com/paper/2602.15531/) (2026)
- [M2LADS Demo: A System for Generating Multimodal Learning Analytics Dashboards](https://mlsanity.com/paper/2502.15363/) (2025)
- [A Multimodal Dataset of Student Oral Presentations with Sensors and Evaluation Data](https://mlsanity.com/paper/2601.07576/) (2026)
- [AIriskEval-edu Demo: Auditing of Pedagogical Risks in Educational Explanations](https://mlsanity.com/paper/2607.25634/) (2026)

## Abstract (arXiv)

This work introduces a new multispectral database and novel approaches for eyeblink detection in RGB and Near-Infrared (NIR) individual images. Our contributed dataset (mEBAL2, multimodal Eye Blink and Attention Level estimation, Version 2) is the largest existing eyeblink database, representing a great opportunity to improve data-driven multispectral approaches for blink detection and related applications (e.g., attention level estimation and presentation attack detection in face biometrics). mEBAL2 includes 21,100 image sequences from 180 different students (more than 2 million labeled images in total) while conducting a number of e-learning tasks of varying difficulty or taking a real course on HTML initiation through the edX MOOC platform. mEBAL2 uses multiple sensors, including two Near-Infrared (NIR) and one RGB camera to capture facial gestures during the execution of the tasks, as well as an Electroencephalogram (EEG) band to get the cognitive activity of the user and blinking events. Furthermore, this work proposes a Convolutional Neural Network architecture as benchmark for blink detection on mEBAL2 with performances up to 97%. Different training methodologies are implemented using the RGB spectrum, NIR spectrum, and the combination of both to enhance the performance on existing eyeblink detectors. We demonstrate that combining NIR and RGB images during training improves the performance of RGB eyeblink detectors (i.e., detection based only on a RGB image). Finally, the generalization capacity of the proposed eyeblink detectors is validated in wilder and more challenging environments like the HUST-LEBW dataset to show the usefulness of mEBAL2 to train a new generation of data-driven approaches for eyeblink detection.