# Neuromorphic Sensing for Yawn Detection in Driver Drowsiness

arXiv: https://arxiv.org/abs/2305.02888 · HTML: https://mlsanity.com/paper/2305.02888/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 17 references that are in the map.
- First posted on arXiv 4 May 2023.

## Builds on (most cited first, 5 of 5)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Real-Time Face & Eye Tracking and Blink Detection using Event Cameras](https://mlsanity.com/paper/2010.08278/) (2020)

## Built on by (most cited first, 3 of 3)

- [Neuromorphic Face Analysis: a Survey](https://mlsanity.com/paper/2402.11631/) (2024)
- [A Lightweight 3D-CNN for Event-Based Human Action Recognition with Privacy-Preserving Potential](https://mlsanity.com/paper/2511.03665/) (2025)
- [Decisive Data using Multi-Modality Optical Sensors for Advanced Vehicular Systems](https://mlsanity.com/paper/2307.13600/) (2023)

## Abstract (arXiv)

Driver monitoring systems (DMS) are a key component of vehicular safety and essential for the transition from semiautonomous to fully autonomous driving. A key task for DMS is to ascertain the cognitive state of a driver and to determine their level of tiredness. Neuromorphic vision systems, based on event camera technology, provide advanced sensing of facial characteristics, in particular the behavior of a driver's eyes. This research explores the potential to extend neuromorphic sensing techniques to analyze the entire facial region, detecting yawning behaviors that give a complimentary indicator of tiredness. A neuromorphic dataset is constructed from 952 video clips (481 yawns, 471 not-yawns) captured with an RGB color camera, with 37 subjects. A total of 95200 neuromorphic image frames are generated from this video data using a video-to-event converter. From these data 21 subjects were selected to provide a training dataset, 8 subjects were used for validation data, and the remaining 8 subjects were reserved for an "unseen" test dataset. An additional 12300 frames were generated from event simulations of a public dataset to test against other methods. A CNN with self-attention and a recurrent head was designed, trained, and tested with these data. Respective precision and recall scores of 95.9 percent and 94.7 percent were achieved on our test set, and 89.9 percent and 91 percent on the simulated public test set, demonstrating the feasibility to add yawn detection as a sensing component of a neuromorphic DMS.