# What's on your mind? A Mental and Perceptual Load Estimation Framework towards Adaptive In-vehicle Interaction while Driving

arXiv: https://arxiv.org/abs/2208.05564 · HTML: https://mlsanity.com/paper/2208.05564/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 110 references that are in the map.
- First posted on arXiv 10 Aug 2022.

## Builds on (most cited first, 5 of 5)

- [Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-based Interaction](https://mlsanity.com/paper/1405.0006/) (2014)
- [You Have a Point There: Object Selection Inside an Automobile Using Gaze, Head Pose and Finger Pointing](https://mlsanity.com/paper/2012.13449/) (2020)
- [ML-PersRef: A Machine Learning-based Personalized Multimodal Fusion Approach for Referencing Outside Objects From a Moving Vehicle](https://mlsanity.com/paper/2111.02327/) (2021)
- [Studying Person-Specific Pointing and Gaze Behavior for Multimodal Referencing of Outside Objects from a Moving Vehicle](https://mlsanity.com/paper/2009.11195/) (2020)
- [Designing Toward Minimalism in Vehicle HMI](https://mlsanity.com/paper/1805.02787/) (2018)

## Built on by (most cited first, 3 of 3)

- [In-Vehicle Interface Adaptation to Environment-Induced Cognitive Workload](https://mlsanity.com/paper/2210.11271/) (2022)
- [Driver Profiling and Bayesian Workload Estimation Using Naturalistic Peripheral Detection Study Data](https://mlsanity.com/paper/2303.14720/) (2023)
- [Adaptive User-centered Neuro-symbolic Learning for Multimodal Interaction with Autonomous Systems](https://mlsanity.com/paper/2309.05787/) (2023)

## Abstract (arXiv)

Several researchers have focused on studying driver cognitive behavior and mental load for in-vehicle interaction while driving. Adaptive interfaces that vary with mental and perceptual load levels could help in reducing accidents and enhancing the driver experience. In this paper, we analyze the effects of mental workload and perceptual load on psychophysiological dimensions and provide a machine learning-based framework for mental and perceptual load estimation in a dual task scenario for in-vehicle interaction (https://github.com/amrgomaaelhady/MWL-PL-estimator). We use off-the-shelf non-intrusive sensors that can be easily integrated into the vehicle's system. Our statistical analysis shows that while mental workload influences some psychophysiological dimensions, perceptual load shows little effect. Furthermore, we classify the mental and perceptual load levels through the fusion of these measurements, moving towards a real-time adaptive in-vehicle interface that is personalized to user behavior and driving conditions. We report up to 89% mental workload classification accuracy and provide a real-time minimally-intrusive solution.