# Real-Time Driver State Monitoring Using a CNN Based Spatio-Temporal Approach

arXiv: https://arxiv.org/abs/1907.08009 · HTML: https://mlsanity.com/paper/1907.08009/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2019.

## Built on by (most cited first, 6 of 6)

- [Vision-Language Models can Identify Distracted Driver Behavior from Naturalistic Videos](https://mlsanity.com/paper/2306.10159/) (2023)
- [A Comparative Analysis of Decision-Level Fusion for Multimodal Driver Behaviour Understanding](https://mlsanity.com/paper/2204.04734/) (2022)
- [Artificial Intelligence Methods in In-Cabin Use Cases: A Survey](https://mlsanity.com/paper/2101.02082/) (2021)
- [Hands-Up: Leveraging Synthetic Data for Hands-On-Wheel Detection](https://mlsanity.com/paper/2206.00148/) (2022)
- [Enhancing Cross-Dataset Performance of Distracted Driving Detection With Score Softmax Classifier And Dynamic Gaussian Smoothing Supervision](https://mlsanity.com/paper/2310.05202/) (2023)
- [DSDFormer: An Innovative Transformer-Mamba Framework for Robust High-Precision Driver Distraction Identification](https://mlsanity.com/paper/2409.05587/) (2024)

## Abstract (arXiv)

Many road accidents occur due to distracted drivers. Today, driver monitoring is essential even for the latest autonomous vehicles to alert distracted drivers in order to take over control of the vehicle in case of emergency. In this paper, a spatio-temporal approach is applied to classify drivers' distraction level and movement decisions using convolutional neural networks (CNNs). We approach this problem as action recognition to benefit from temporal information in addition to spatial information. Our approach relies on features extracted from sparsely selected frames of an action using a pre-trained BN-Inception network. Experiments show that our approach outperforms the state-of-the art results on the Distracted Driver Dataset (96.31%), with an accuracy of 99.10% for 10-class classification while providing real-time performance. We also analyzed the impact of fusion using RGB and optical flow modalities with a very recent data level fusion strategy. The results on the Distracted Driver and Brain4Cars datasets show that fusion of these modalities further increases the accuracy.