mlsanityData to 14 Oct 2026

Safe Control Transitions: Machine Vision Based Observable Readiness Index and Data-Driven Takeover Time Prediction

arXiv 2301.05805 ↗First posted 14 Jan 2023cs.CV cs.AI cs.HCcited by 12

Not on the map. Why?

Abstract (arXiv)

To make safe transitions from autonomous to manual control, a vehicle must have a representation of the awareness of driver state; two metrics which quantify this state are the Observable Readiness Index and Takeover Time. In this work, we show that machine learning models which predict these two metrics are robust to multiple camera views, expanding from the limited view angles in prior research. Importantly, these models take as input feature vectors corresponding to hand location and activity as well as gaze location, and we explore the tradeoffs of different views in generating these feature vectors. Further, we introduce two metrics to evaluate the quality of control transitions following the takeover event (the maximal lateral deviation and velocity deviation) and compute correlations of these post-takeover metrics to the pre-takeover predictive metrics.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 16 of its 33 references that are in the map. First posted on arXiv 14 Jan 2023.

Builds on

16 of 33 references.

References · 16

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  6. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  7. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  8. 2020Trajectory Prediction in Autonomous Driving with a Lane Heading Auxiliary Losscited by 30
  9. 2018Driver Gaze Zone Estimation using Convolutional Neural Networks: A General Framework and Ablative Analysiscited by 20
  10. 2018Looking at the Driver/Rider in Autonomous Vehicles to Predict Take-Over Readinesscited by 13
  11. 2021Autonomous Vehicles that Alert Humans to Take-Over Controls: Modeling with Real-World Datacited by 13
  12. 2021Predicting Take-over Time for Autonomous Driving with Real-World Data: Robust Data Augmentation, Models, and Evaluationcited by 10
4 more
  1. 2018Looking at Hands in Autonomous Vehicles: A ConvNet Approach using Part Affinity Fieldscited by 7
  2. 2018HandyNet: A One-stop Solution to Detect, Segment, Localize & Analyze Driver Handscited by 6
  3. 2019Forced Spatial Attention for Driver Foot Activity Classificationcited by 4
  4. 2022From Pedestrian Detection to Crosswalk Estimation: An EM Algorithm and Analysis on Diverse Datasetscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 6; 2025: 3; 2026: 1.

12 papers.

  1. 2024Towards Explainable, Safe Autonomous Driving with Language Embeddings for Novelty Identification and Active Learning: Framework and Experimental Analysis with Real-World Data Setscited by 25
  2. 2024doScenes: An Autonomous Driving Dataset with Natural Language Instruction for Human Interaction and Vision-Language Navigationcited by 15
  3. 2023Ensemble Learning for Fusion of Multiview Vision with Occlusion and Missing Information: Framework and Evaluations with Real-World Data and Applications in Driver Hand Activity Recognitioncited by 4
  4. 2024Driver Activity Classification Using Generalizable Representations from Vision-Language Modelscited by 3
  5. 2024Evaluating Driver Readiness in Conditionally Automated Vehicles from Eye-Tracking Data and Head Posecited by 2
  6. 2024Learning to Find Missing Video Frames with Synthetic Data Augmentation: A General Framework and Application in Generating Thermal Images Using RGB Camerascited by 2
  7. 2025Towards Vision Zero: The TUM Traffic Accid3nD Datasetcited by 2
  8. 2025Enhancing Highway Safety: Accident Detection on the A9 Test Stretch Using Roadside Sensorscited by 1
  9. 2025Safety-Critical Learning for Long-Tail Events: The TUM Traffic Accident Datasetcited by 1
  10. 2023Vision-based Analysis of Driver Activity and Driving Performance Under the Influence of Alcoholcited by 0
  11. 2024Creativity and Visual Communication from Machine to Musician: Sharing a Score through a Robotic Cameracited by 0
  12. 2026Looking and Listening Inside and Outside: Multimodal Artificial Intelligence Systems for Driver Safety Assessment and Intelligent Vehicle Decision-Makingcited by 0