# MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning

arXiv: https://arxiv.org/abs/1912.07773 · HTML: https://mlsanity.com/paper/1912.07773/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 31 of its 76 references that are in the map.
- First posted on arXiv 17 Dec 2019.

## Placed in

- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [Exploring the Limitations of Behavior Cloning for Autonomous Driving](https://mlsanity.com/paper/1904.08980/) (2019)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Visual Saliency Based on Multiscale Deep Features](https://mlsanity.com/paper/1503.08663/) (2015)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1507.04888/) (2015)
- [End-to-end Learning of Driving Models from Large-scale Video Datasets](https://mlsanity.com/paper/1612.01079/) (2016)
- [Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention](https://mlsanity.com/paper/1703.10631/) (2017)
- [On Learning Intrinsic Rewards for Policy Gradient Methods](https://mlsanity.com/paper/1804.06459/) (2018)
- [Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoning](https://mlsanity.com/paper/1811.02307/) (2018)
- [Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model](https://mlsanity.com/paper/1611.09571/) (2016)
- [Predicting the Driver's Focus of Attention: the DR(eye)VE Project](https://mlsanity.com/paper/1705.03854/) (2017)
- [Deep Visual Attention Prediction](https://mlsanity.com/paper/1705.02544/) (2017)
- [Revisiting Video Saliency: A Large-scale Benchmark and a New Model](https://mlsanity.com/paper/1801.07424/) (2018)
- [Predicting Driver Attention in Critical Situations](https://mlsanity.com/paper/1711.06406/) (2017)
- [TASED-Net: Temporally-Aggregating Spatial Encoder-Decoder Network for Video Saliency Detection](https://mlsanity.com/paper/1908.05786/) (2019)
- [DADA-2000: Can Driving Accident be Predicted by Driver Attention? Analyzed by A Benchmark](https://mlsanity.com/paper/1904.12634/) (2019)
- [VPGNet: Vanishing Point Guided Network for Lane and Road Marking Detection and Recognition](https://mlsanity.com/paper/1710.06288/) (2017)
- [CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research](https://mlsanity.com/paper/1505.03581/) (2015)
- [DeepFix: A Fully Convolutional Neural Network for predicting Human Eye Fixations](https://mlsanity.com/paper/1510.02927/) (2015)
- [Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition](https://mlsanity.com/paper/1312.7570/) (2013)
- [AGIL: Learning Attention from Human for Visuomotor Tasks](https://mlsanity.com/paper/1806.03960/) (2018)
- [Looking at the right stuff: Guided semantic-gaze for autonomous driving](https://mlsanity.com/paper/1911.10455/) (2019)
- [Periphery-Fovea Multi-Resolution Driving Model guided by Human Attention](https://mlsanity.com/paper/1903.09950/) (2019)
- [Goal-oriented Object Importance Estimation in On-road Driving Videos](https://mlsanity.com/paper/1905.02848/) (2019)
- [Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraints](https://mlsanity.com/paper/1906.00429/) (2019)

## Built on by (most cited first, 29 of 29)

- [Nexar Dashcam Collision Prediction Dataset and Challenge](https://mlsanity.com/paper/2503.03848/) (2025)
- [Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embedding](https://mlsanity.com/paper/2303.09706/) (2023)
- [Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmark](https://mlsanity.com/paper/2212.09381/) (2022)
- [FBLNet: FeedBack Loop Network for Driver Attention Prediction](https://mlsanity.com/paper/2212.02096/) (2022)
- [RiskBench: A Scenario-based Benchmark for Risk Identification](https://mlsanity.com/paper/2312.01659/) (2023)
- [Where, What, Why: Towards Explainable Driver Attention Prediction](https://mlsanity.com/paper/2506.23088/) (2025)
- [Behavioral Research and Practical Models of Drivers' Attention](https://mlsanity.com/paper/2104.05677/) (2021)
- [DROID: Driver-centric Risk Object Identification](https://mlsanity.com/paper/2106.13201/) (2021)
- [MAAD: A Model and Dataset for "Attended Awareness" in Driving](https://mlsanity.com/paper/2110.08610/) (2021)
- [Trends, Applications, and Challenges in Human Attention Modelling](https://mlsanity.com/paper/2402.18673/) (2024)
- [CoCAtt: A Cognitive-Conditioned Driver Attention Dataset](https://mlsanity.com/paper/2111.10014/) (2021)
- [SCOUT+: Towards Practical Task-Driven Drivers' Gaze Prediction](https://mlsanity.com/paper/2404.08756/) (2024)
- [Understanding and Modeling the Effects of Task and Context on Drivers' Gaze Allocation](https://mlsanity.com/paper/2310.09275/) (2023)
- [SalM2: An Extremely Lightweight Saliency Mamba Model for Real-Time Cognitive Awareness of Driver Attention](https://mlsanity.com/paper/2502.16214/) (2025)
- [Data Limitations for Modeling Top-Down Effects on Drivers' Attention](https://mlsanity.com/paper/2404.08749/) (2024)
- [Learning Implicit Social Navigation Behavior using Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/2501.06946/) (2025)
- [Gated Driver Attention Predictor](https://mlsanity.com/paper/2308.02530/) (2023)
- [Potential Field as Scene Affordance for Behavior Change-Based Visual Risk Object Identification](https://mlsanity.com/paper/2409.15846/) (2024)
- [AEGIS: Human Attention-based Explainable Guidance for Intelligent Vehicle Systems](https://mlsanity.com/paper/2504.05950/) (2025)
- [Causal-Entity Reflected Egocentric Traffic Accident Video Synthesis](https://mlsanity.com/paper/2506.23263/) (2025)
- [CUEING: a lightweight model to Capture hUman attEntion In driviNG](https://mlsanity.com/paper/2305.15710/) (2023)
- [DriverGaze360: OmniDirectional Driver Attention with Object-Level Guidance](https://mlsanity.com/paper/2512.14266/) (2025)
- [From Scene to Object: Text-Guided Dual-Gaze Prediction](https://mlsanity.com/paper/2604.20191/) (2026)
- [Evaluating the Perceived Safety of Urban City via Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/2211.10660/) (2022)
- [Gaze Prediction as a Function of Eye Movement Type and Individual Differences](https://mlsanity.com/paper/2501.00597/) (2024)
- [Towards Robust Unsupervised Attention Prediction in Autonomous Driving](https://mlsanity.com/paper/2501.15045/) (2025)
- [Active Sensing Shapes Real-World Decision-Making through Dynamic Evidence Accumulation](https://mlsanity.com/paper/2601.04214/) (2025)
- [Uncertainty-Aware Vision-based Risk Object Identification via Conformal Risk Tube Prediction](https://mlsanity.com/paper/2603.23919/) (2026)
- [DiffAttn: Diffusion-Based Drivers' Visual Attention Prediction with LLM-Enhanced Semantic Reasoning](https://mlsanity.com/paper/2603.28251/) (2026)

## Abstract (arXiv)

Inspired by human visual attention, we propose a novel inverse reinforcement learning formulation using Maximum Entropy Deep Inverse Reinforcement Learning (MEDIRL) for predicting the visual attention of drivers in accident-prone situations. MEDIRL predicts fixation locations that lead to maximal rewards by learning a task-sensitive reward function from eye fixation patterns recorded from attentive drivers. Additionally, we introduce EyeCar, a new driver attention dataset in accident-prone situations. We conduct comprehensive experiments to evaluate our proposed model on three common benchmarks: (DR(eye)VE, BDD-A, DADA-2000), and our EyeCar dataset. Results indicate that MEDIRL outperforms existing models for predicting attention and achieves state-of-the-art performance. We present extensive ablation studies to provide more insights into different features of our proposed model.