# DMD: A Large-Scale Multi-Modal Driver Monitoring Dataset for Attention and Alertness Analysis

arXiv: https://arxiv.org/abs/2008.12085 · HTML: https://mlsanity.com/paper/2008.12085/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2020.

## Built on by (most cited first, 30 of 47)

- [FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integration](https://mlsanity.com/paper/2108.06428/) (2021)
- [AIDE: A Vision-Driven Multi-View, Multi-Modal, Multi-Tasking Dataset for Assistive Driving Perception](https://mlsanity.com/paper/2307.13933/) (2023)
- [IC3M: In-Car Multimodal Multi-object Monitoring for Abnormal Status of Both Driver and Passengers](https://mlsanity.com/paper/2410.02592/) (2024)
- [Vision-Language Models can Identify Distracted Driver Behavior from Naturalistic Videos](https://mlsanity.com/paper/2306.10159/) (2023)
- [A Review of Driver Gaze Estimation and Application in Gaze Behavior Understanding](https://mlsanity.com/paper/2307.01470/) (2023)
- [Efficient Mixture-of-Expert for Video-based Driver State and Physiological Multi-task Estimation in Conditional Autonomous Driving](https://mlsanity.com/paper/2410.21086/) (2024)
- [Supervised Contrastive Learning for Detecting Anomalous Driving Behaviours from Multimodal Videos](https://mlsanity.com/paper/2109.04021/) (2021)
- [PDB: Not All Drivers Are the Same -- A Personalized Dataset for Understanding Driving Behavior](https://mlsanity.com/paper/2503.06477/) (2025)
- [Real-Time Driver Monitoring Systems through Modality and View Analysis](https://mlsanity.com/paper/2210.09441/) (2022)
- [Visual Dominance and Emerging Multimodal Approaches in Distracted Driving Detection: A Review of Machine Learning Techniques](https://mlsanity.com/paper/2505.01973/) (2025)
- [Soap2Soap: Long Cinematic Video Remaking via Multi-Agent Collaboration](https://mlsanity.com/paper/2605.17423/) (2026)
- [Is my Driver Observation Model Overconfident? Input-guided Calibration Networks for Reliable and Interpretable Confidence Estimates](https://mlsanity.com/paper/2204.04674/) (2022)
- [Spiking-DD: Neuromorphic Event Camera based Driver Distraction Detection with Spiking Neural Network](https://mlsanity.com/paper/2407.20633/) (2024)
- [A Survey on Drowsiness Detection -- Modern Applications and Methods](https://mlsanity.com/paper/2408.12990/) (2024)
- [Hands-Up: Leveraging Synthetic Data for Hands-On-Wheel Detection](https://mlsanity.com/paper/2206.00148/) (2022)
- [Leveraging Driver Field-of-View for Multimodal Ego-Trajectory Prediction](https://mlsanity.com/paper/2312.08558/) (2023)
- [D2E-An Autonomous Decision-making Dataset involving Driver States and Human Evaluation](https://mlsanity.com/paper/2406.01598/) (2024)
- [Risk-Aware Selective Multimodal Driver Monitoring with Driver-State World Modeling](https://mlsanity.com/paper/2606.26922/) (2026)
- [PO-GUISE+: Pose and object guided transformer token selection for efficient driver action recognition](https://mlsanity.com/paper/2407.13750/) (2024)
- [myEye2Wheeler: A Two-Wheeler Indian Driver Real-World Eye-Tracking Dataset](https://mlsanity.com/paper/2502.12723/) (2025)
- [Sage Deer: A Super-Aligned Driving Generalist Is Your Copilot](https://mlsanity.com/paper/2505.10257/) (2025)
- [Driver-Net: Multi-Camera Fusion for Assessing Driver Take-Over Readiness in Automated Vehicles](https://mlsanity.com/paper/2507.04139/) (2025)
- [MILD: Mediator Agent System with Bidirectional Perception and Multi-Layered Alignment for Human-Vehicle Collaboration](https://mlsanity.com/paper/2605.01507/) (2026)
- [Applying Spatiotemporal Attention to Identify Distracted and Drowsy Driving with Vision Transformers](https://mlsanity.com/paper/2207.12148/) (2022)
- [Robust Multiview Multimodal Driver Monitoring System Using Masked Multi-Head Self-Attention](https://mlsanity.com/paper/2304.06370/) (2023)
- [Quantized Distillation: Optimizing Driver Activity Recognition Models for Resource-Constrained Environments](https://mlsanity.com/paper/2311.05970/) (2023)
- [A Review Paper of the Effects of Distinct Modalities and ML Techniques to Distracted Driving Detection](https://mlsanity.com/paper/2501.11758/) (2025)
- [UL-DD: A Multimodal Drowsiness Dataset Using Video, Biometric Signals, and Behavioral Data](https://mlsanity.com/paper/2507.13403/) (2025)
- [SGAP-Gaze: Scene Grid Attention Based Point-of-Gaze Estimation Network for Driver Gaze](https://mlsanity.com/paper/2604.19888/) (2026)
- [DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecasting](https://mlsanity.com/paper/2609.08117/) (2026)

## Abstract (arXiv)

Vision is the richest and most cost-effective technology for Driver Monitoring Systems (DMS), especially after the recent success of Deep Learning (DL) methods. The lack of sufficiently large and comprehensive datasets is currently a bottleneck for the progress of DMS development, crucial for the transition of automated driving from SAE Level-2 to SAE Level-3. In this paper, we introduce the Driver Monitoring Dataset (DMD), an extensive dataset which includes real and simulated driving scenarios: distraction, gaze allocation, drowsiness, hands-wheel interaction and context data, in 41 hours of RGB, depth and IR videos from 3 cameras capturing face, body and hands of 37 drivers. A comparison with existing similar datasets is included, which shows the DMD is more extensive, diverse, and multi-purpose. The usage of the DMD is illustrated by extracting a subset of it, the dBehaviourMD dataset, containing 13 distraction activities, prepared to be used in DL training processes. Furthermore, we propose a robust and real-time driver behaviour recognition system targeting a real-world application that can run on cost-efficient CPU-only platforms, based on the dBehaviourMD. Its performance is evaluated with different types of fusion strategies, which all reach enhanced accuracy still providing real-time response.