mlsanityData to 14 Oct 2026

MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning

arXiv 1912.07773 ↗First posted 17 Dec 2019cs.CV cs.HC cs.LGcited by 29

On the map: Offline RL and imitation learning from data → Imitation and offline RL for driving

Abstract (arXiv)

Inspired by human visual attention, we propose a novel inverse reinforcement learning formulation using Maximum Entropy Deep Inverse Reinforcement Learning (MEDIRL) for predicting the visual attention of drivers in accident-prone situations. MEDIRL predicts fixation locations that lead to maximal rewards by learning a task-sensitive reward function from eye fixation patterns recorded from attentive drivers. Additionally, we introduce EyeCar, a new driver attention dataset in accident-prone situations. We conduct comprehensive experiments to evaluate our proposed model on three common benchmarks: (DR(eye)VE, BDD-A, DADA-2000), and our EyeCar dataset. Results indicate that MEDIRL outperforms existing models for predicting attention and achieves state-of-the-art performance. We present extensive ablation studies to provide more insights into different features of our proposed model.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 76 references that are in the map. First posted on arXiv 17 Dec 2019.

Builds on

31 of 76 references.

References · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
  3. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  4. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  5. 2019Video Instance Segmentationcited by 393
  6. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 346
  7. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 231
  8. 2015Visual Saliency Based on Multiscale Deep Featurescited by 185
  9. 2016What do different evaluation metrics tell us about saliency models?cited by 185
  10. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 176
  11. 2016End-to-end Learning of Driving Models from Large-scale Video Datasetscited by 148
  12. 2017Interpretable Learning for Self-Driving Cars by Visualizing Causal Attentioncited by 107
19 more
  1. 2018On Learning Intrinsic Rewards for Policy Gradient Methodscited by 102
  2. 2018Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoningcited by 102
  3. 2016Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Modelcited by 91
  4. 2017Predicting the Driver's Focus of Attention: the DR(eye)VE Projectcited by 79
  5. 2017Deep Visual Attention Predictioncited by 70
  6. 2018Revisiting Video Saliency: A Large-scale Benchmark and a New Modelcited by 67
  7. 2017Predicting Driver Attention in Critical Situationscited by 61
  8. 2019TASED-Net: Temporally-Aggregating Spatial Encoder-Decoder Network for Video Saliency Detectioncited by 59
  9. 2019DADA-2000: Can Driving Accident be Predicted by Driver Attention? Analyzed by A Benchmarkcited by 57
  10. 2017VPGNet: Vanishing Point Guided Network for Lane and Road Marking Detection and Recognitioncited by 56
  11. 2015CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Researchcited by 52
  12. 2015DeepFix: A Fully Convolutional Neural Network for predicting Human Eye Fixationscited by 48
  13. 2013Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognitioncited by 37
  14. 2018AGIL: Learning Attention from Human for Visuomotor Taskscited by 32
  15. 2019Looking at the right stuff: Guided semantic-gaze for autonomous drivingcited by 32
  16. 2019Periphery-Fovea Multi-Resolution Driving Model guided by Human Attentioncited by 19
  17. 2019Goal-oriented Object Importance Estimation in On-road Driving Videoscited by 19
  18. 2019Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraintscited by 18
  19. 2017Predicting Video Saliency with Object-to-Motion CNN and Two-layer Convolutional LSTMcited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 3; 2023: 5; 2024: 5; 2025: 9; 2026: 3.

29 papers.

Sort:

By area:

  1. 2025Nexar Dashcam Collision Prediction Dataset and Challengecited by 21
  2. 2023Unsupervised Self-Driving Attention Prediction via Uncertainty Mining and Knowledge Embeddingcited by 19
  3. 2022Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmarkcited by 16
  4. 2022FBLNet: FeedBack Loop Network for Driver Attention Predictioncited by 12
  5. 2023RiskBench: A Scenario-based Benchmark for Risk Identificationcited by 12
  6. 2025Where, What, Why: Towards Explainable Driver Attention Predictioncited by 10
  7. 2021Behavioral Research and Practical Models of Drivers' Attentioncited by 9
  8. 2021DROID: Driver-centric Risk Object Identificationcited by 9
  9. 2021MAAD: A Model and Dataset for "Attended Awareness" in Drivingcited by 9
  10. 2024Trends, Applications, and Challenges in Human Attention Modellingcited by 9
  11. 2021CoCAtt: A Cognitive-Conditioned Driver Attention Datasetcited by 8
  12. 2024SCOUT+: Towards Practical Task-Driven Drivers' Gaze Predictioncited by 5
17 more
  1. 2023Understanding and Modeling the Effects of Task and Context on Drivers' Gaze Allocationcited by 4
  2. 2025SalM2: An Extremely Lightweight Saliency Mamba Model for Real-Time Cognitive Awareness of Driver Attentioncited by 4
  3. 2024Data Limitations for Modeling Top-Down Effects on Drivers' Attentioncited by 3
  4. 2025Learning Implicit Social Navigation Behavior using Deep Inverse Reinforcement Learningcited by 3
  5. 2023Gated Driver Attention Predictorcited by 2
  6. 2024Potential Field as Scene Affordance for Behavior Change-Based Visual Risk Object Identificationcited by 2
  7. 2025AEGIS: Human Attention-based Explainable Guidance for Intelligent Vehicle Systemscited by 2
  8. 2025Causal-Entity Reflected Egocentric Traffic Accident Video Synthesiscited by 2
  9. 2023CUEING: a lightweight model to Capture hUman attEntion In driviNGcited by 1
  10. 2025DriverGaze360: OmniDirectional Driver Attention with Object-Level Guidancecited by 1
  11. 2026From Scene to Object: Text-Guided Dual-Gaze Predictioncited by 1
  12. 2022Evaluating the Perceived Safety of Urban City via Maximum Entropy Deep Inverse Reinforcement Learningcited by 0
  13. 2024Gaze Prediction as a Function of Eye Movement Type and Individual Differencescited by 0
  14. 2025Towards Robust Unsupervised Attention Prediction in Autonomous Drivingcited by 0
  15. 2025Active Sensing Shapes Real-World Decision-Making through Dynamic Evidence Accumulationcited by 0
  16. 2026Uncertainty-Aware Vision-based Risk Object Identification via Conformal Risk Tube Predictioncited by 0
  17. 2026DiffAttn: Diffusion-Based Drivers' Visual Attention Prediction with LLM-Enhanced Semantic Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning
  2. 2024Evaluating MEDIRL: A Replication and Ablation Study of Maximum Entropy Deep Inverse Reinforcement Learning for Human Social Navigation
  3. 2025Estimating cognitive biases with attention-aware inverse planning
  4. 2020Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Driving
  5. 2020Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learning
  6. 2026Game-Theoretic Inverse Reinforcement Learning for Modeling Competitive Human Driving: A Cut-in Prediction Study
  7. 2022Driving in Real Life with Inverse Reinforcement Learning
  8. 2022Spatiotemporal Costmap Inference for MPC via Deep Inverse Reinforcement Learning