# Predicting the Driver's Focus of Attention: the DR(eye)VE Project

arXiv: https://arxiv.org/abs/1705.03854 · HTML: https://mlsanity.com/paper/1705.03854/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2017.

## Built on by (most cited first, 30 of 79)

- [EgoLife: Towards Egocentric Life Assistant](https://mlsanity.com/paper/2503.03803/) (2025)
- [Appearance-based Gaze Estimation With Deep Learning: A Review and Benchmark](https://mlsanity.com/paper/2104.12668/) (2021)
- [A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlook](https://mlsanity.com/paper/2401.01454/) (2024)
- [Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review](https://mlsanity.com/paper/2402.10086/) (2024)
- [MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1912.07773/) (2019)
- [Towards Knowledge-driven Autonomous Driving](https://mlsanity.com/paper/2312.04316/) (2023)
- [Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embedding](https://mlsanity.com/paper/2303.09706/) (2023)
- [Learning from Observer Gaze:Zero-Shot Attention Prediction Oriented by Human-Object Interaction Recognition](https://mlsanity.com/paper/2405.09931/) (2024)
- [Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmark](https://mlsanity.com/paper/2212.09381/) (2022)
- [Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks](https://mlsanity.com/paper/2107.05825/) (2021)
- [Important Object Identification with Semi-Supervised Learning for Autonomous Driving](https://mlsanity.com/paper/2203.02634/) (2022)
- [FBLNet: FeedBack Loop Network for Driver Attention Prediction](https://mlsanity.com/paper/2212.02096/) (2022)
- [A Review of Driver Gaze Estimation and Application in Gaze Behavior Understanding](https://mlsanity.com/paper/2307.01470/) (2023)
- [Gaze Training by Modulated Dropout Improves Imitation Learning](https://mlsanity.com/paper/1904.08377/) (2019)
- [Where, What, Why: Towards Explainable Driver Attention Prediction](https://mlsanity.com/paper/2506.23088/) (2025)
- [MAAD: A Model and Dataset for "Attended Awareness" in Driving](https://mlsanity.com/paper/2110.08610/) (2021)
- [Trends, Applications, and Challenges in Human Attention Modelling](https://mlsanity.com/paper/2402.18673/) (2024)
- [CoCAtt: A Cognitive-Conditioned Driver Attention Dataset](https://mlsanity.com/paper/2111.10014/) (2021)
- [Visual Attention Prediction Improves Performance of Autonomous Drone Racing Agents](https://mlsanity.com/paper/2201.02569/) (2022)
- [Feature-Aligned Video Raindrop Removal with Temporal Constraints](https://mlsanity.com/paper/2205.14574/) (2022)
- [Video Waterdrop Removal via Spatio-Temporal Fusion in Driving Scenes](https://mlsanity.com/paper/2302.05916/) (2023)
- [M2DA: Multi-Modal Fusion Transformer Incorporating Driver Attention for Autonomous Driving](https://mlsanity.com/paper/2403.12552/) (2024)
- [Selective Eye-gaze Augmentation To Enhance Imitation Learning In Atari Games](https://mlsanity.com/paper/2012.03145/) (2020)
- [PDB: Not All Drivers Are the Same -- A Personalized Dataset for Understanding Driving Behavior](https://mlsanity.com/paper/2503.06477/) (2025)
- [SCOUT+: Towards Practical Task-Driven Drivers' Gaze Prediction](https://mlsanity.com/paper/2404.08756/) (2024)
- [Learning Road Scene-level Representations via Semantic Region Prediction](https://mlsanity.com/paper/2301.00714/) (2023)
- [Understanding and Modeling the Effects of Task and Context on Drivers' Gaze Allocation](https://mlsanity.com/paper/2310.09275/) (2023)
- [What Do Deep Saliency Models Learn about Visual Attention?](https://mlsanity.com/paper/2310.09679/) (2023)
- [Artificial Intelligence Methods in In-Cabin Use Cases: A Survey](https://mlsanity.com/paper/2101.02082/) (2021)
- [A Gated Fusion Network for Dynamic Saliency Prediction](https://mlsanity.com/paper/2102.07682/) (2021)

## Abstract (arXiv)

In this work we aim to predict the driver's focus of attention. The goal is to estimate what a person would pay attention to while driving, and which part of the scene around the vehicle is more critical for the task. To this end we propose a new computer vision model based on a multi-branch deep architecture that integrates three sources of information: raw video, motion and scene semantics. We also introduce DR(eye)VE, the largest dataset of driving scenes for which eye-tracking annotations are available. This dataset features more than 500,000 registered frames, matching ego-centric views (from glasses worn by drivers) and car-centric views (from roof-mounted camera), further enriched by other sensors measurements. Results highlight that several attention patterns are shared across drivers and can be reproduced to some extent. The indication of which elements in the scene are likely to capture the driver's attention may benefit several applications in the context of human-vehicle interaction and driver attention analysis.