# Looking at the Driver/Rider in Autonomous Vehicles to Predict Take-Over Readiness

arXiv: https://arxiv.org/abs/1811.06047 · HTML: https://mlsanity.com/paper/1811.06047/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2018.

## Built on by (most cited first, 13 of 13)

- [TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration](https://mlsanity.com/paper/2203.00927/) (2022)
- [Autonomous Vehicles that Alert Humans to Take-Over Controls: Modeling with Real-World Data](https://mlsanity.com/paper/2104.11489/) (2021)
- [Safe Control Transitions: Machine Vision Based Observable Readiness Index and Data-Driven Takeover Time Prediction](https://mlsanity.com/paper/2301.05805/) (2023)
- [Predicting Take-over Time for Autonomous Driving with Real-World Data: Robust Data Augmentation, Models, and Evaluation](https://mlsanity.com/paper/2107.12932/) (2021)
- [A Spatio-Temporal Multilayer Perceptron for Gesture Recognition](https://mlsanity.com/paper/2204.11511/) (2022)
- [Is my Driver Observation Model Overconfident? Input-guided Calibration Networks for Reliable and Interpretable Confidence Estimates](https://mlsanity.com/paper/2204.04674/) (2022)
- [(Safe) SMART Hands: Hand Activity Analysis and Distraction Alerts Using a Multi-Camera Framework](https://mlsanity.com/paper/2301.05838/) (2023)
- [Ensemble Learning for Fusion of Multiview Vision with Occlusion and Missing Information: Framework and Evaluations with Real-World Data and Applications in Driver Hand Activity Recognition](https://mlsanity.com/paper/2301.12592/) (2023)
- [Evaluating Driver Readiness in Conditionally Automated Vehicles from Eye-Tracking Data and Head Pose](https://mlsanity.com/paper/2401.11284/) (2024)
- [Driver-Net: Multi-Camera Fusion for Assessing Driver Take-Over Readiness in Automated Vehicles](https://mlsanity.com/paper/2507.04139/) (2025)
- [Research on target detection method of distracted driving behavior based on improved YOLOv8](https://mlsanity.com/paper/2407.01864/) (2024)
- [Toward Informed AV Decision-Making: Computational Model of Well-being and Trust in Mobility](https://mlsanity.com/paper/2505.14983/) (2025)
- [Looking and Listening Inside and Outside: Multimodal Artificial Intelligence Systems for Driver Safety Assessment and Intelligent Vehicle Decision-Making](https://mlsanity.com/paper/2602.07668/) (2026)

## Abstract (arXiv)

Continuous estimation the driver's take-over readiness is critical for safe and timely transfer of control during the failure modes of autonomous vehicles. In this paper, we propose a data-driven approach for estimating the driver's take-over readiness based purely on observable cues from in-vehicle vision sensors. We present an extensive naturalistic drive dataset of drivers in a conditionally autonomous vehicle running on Californian freeways. We collect subjective ratings for the driver's take-over readiness from multiple human observers viewing the sensor feed. Analysis of the ratings in terms of intra-class correlation coefficients (ICCs) shows a high degree of consistency in the ratings across raters. We define a metric for the driver's take-over readiness termed the 'Observable Readiness Index (ORI)' based on the ratings. Finally, we propose an LSTM model for continuous estimation of the driver's ORI based on a holistic representation of the driver's state, capturing gaze, hand, pose and foot activity. Our model estimates the ORI with a mean absolute error of 0.449 on a 5 point scale.