# UL-DD: A Multimodal Drowsiness Dataset Using Video, Biometric Signals, and Behavioral Data

arXiv: https://arxiv.org/abs/2507.13403 · HTML: https://mlsanity.com/paper/2507.13403/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 72 references that are in the map.
- First posted on arXiv 16 Jul 2025.

## Builds on (most cited first, 10 of 10)

- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [DMD: A Large-Scale Multi-Modal Driver Monitoring Dataset for Attention and Alertness Analysis](https://mlsanity.com/paper/2008.12085/) (2020)
- [A Multimodal Approach to Estimating Vigilance Using EEG and Forehead EOG](https://mlsanity.com/paper/1611.08492/) (2016)
- [The Multimodal Sentiment Analysis in Car Reviews (MuSe-CaR) Dataset: Collection, Insights and Improvements](https://mlsanity.com/paper/2101.06053/) (2021)
- [A Realistic Dataset and Baseline Temporal Model for Early Drowsiness Detection](https://mlsanity.com/paper/1904.07312/) (2019)
- [A Multimodal Intermediate Fusion Network with Manifold Learning for Stress Detection](https://mlsanity.com/paper/2403.08077/) (2024)
- [A Survey of Wearable Devices Pairing Based on Biometric Signals](https://mlsanity.com/paper/2107.11685/) (2021)
- [Multimodal Stress Detection Using Facial Landmarks and Biometric Signals](https://mlsanity.com/paper/2311.03606/) (2023)
- [Drowsiness Detection Based On Driver Temporal Behavior Using a New Developed Dataset](https://mlsanity.com/paper/2104.00125/) (2021)

## Built on by (most cited first, 1 of 1)

- [Blinking Beyond EAR: A Stable Eyelid Angle Metric for Driver Drowsiness Detection and Data Augmentation](https://mlsanity.com/paper/2511.19519/) (2025)

## Abstract (arXiv)

In this study, we present a comprehensive public dataset for driver drowsiness detection, integrating multimodal signals of facial, behavioral, and biometric indicators. Our dataset includes 3D facial video using a depth camera, IR camera footage, posterior videos, and biometric signals such as heart rate, electrodermal activity, blood oxygen saturation, skin temperature, and accelerometer data. This data set provides grip sensor data from the steering wheel and telemetry data from the American truck simulator game to provide more information about drivers' behavior while they are alert and drowsy. Drowsiness levels were self-reported every four minutes using the Karolinska Sleepiness Scale (KSS). The simulation environment consists of three monitor setups, and the driving condition is completely like a car. Data were collected from 19 subjects (15 M, 4 F) in two conditions: when they were fully alert and when they exhibited signs of sleepiness. Unlike other datasets, our multimodal dataset has a continuous duration of 40 minutes for each data collection session per subject, contributing to a total length of 1,400 minutes, and we recorded gradual changes in the driver state rather than discrete alert/drowsy labels. This study aims to create a comprehensive multimodal dataset of driver drowsiness that captures a wider range of physiological, behavioral, and driving-related signals. The dataset will be available upon request to the corresponding author.