# MAAD: A Model and Dataset for "Attended Awareness" in Driving

arXiv: https://arxiv.org/abs/2110.08610 · HTML: https://mlsanity.com/paper/2110.08610/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 68 references that are in the map.
- First posted on arXiv 16 Oct 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Panoptic Segmentation](https://mlsanity.com/paper/1801.00868/) (2018)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [Regularization With Stochastic Transformations and Perturbations for Deep Semi-Supervised Learning](https://mlsanity.com/paper/1606.04586/) (2016)
- [Gaze360: Physically Unconstrained Gaze Estimation in the Wild](https://mlsanity.com/paper/1910.10088/) (2019)
- [Salient Object Detection: A Benchmark](https://mlsanity.com/paper/1501.02741/) (2015)
- [Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model](https://mlsanity.com/paper/1611.09571/) (2016)
- [Predicting the Driver's Focus of Attention: the DR(eye)VE Project](https://mlsanity.com/paper/1705.03854/) (2017)
- [Predicting Driver Attention in Critical Situations](https://mlsanity.com/paper/1711.06406/) (2017)
- [TASED-Net: Temporally-Aggregating Spatial Encoder-Decoder Network for Video Saliency Detection](https://mlsanity.com/paper/1908.05786/) (2019)
- [A Deep Multi-Level Network for Saliency Prediction](https://mlsanity.com/paper/1609.01064/) (2016)
- [Shallow and Deep Convolutional Networks for Saliency Prediction](https://mlsanity.com/paper/1603.00845/) (2016)
- [STAViS: Spatio-Temporal AudioVisual Saliency Network](https://mlsanity.com/paper/2001.03063/) (2020)
- [Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition](https://mlsanity.com/paper/1312.7570/) (2013)
- [MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1912.07773/) (2019)
- [Recurrent Mixture Density Network for Spatiotemporal Visual Attention](https://mlsanity.com/paper/1603.08199/) (2016)
- [Classifying Eye-Tracking Data Using Saliency Maps](https://mlsanity.com/paper/2010.12913/) (2020)

## Built on by (most cited first, 9 of 9)

- [Understanding and Modeling the Effects of Task and Context on Drivers' Gaze Allocation](https://mlsanity.com/paper/2310.09275/) (2023)
- [Leveraging Driver Field-of-View for Multimodal Ego-Trajectory Prediction](https://mlsanity.com/paper/2312.08558/) (2023)
- [Data Limitations for Modeling Top-Down Effects on Drivers' Attention](https://mlsanity.com/paper/2404.08749/) (2024)
- [Enhancing End-to-End Autonomous Driving Systems Through Synchronized Human Behavior Data](https://mlsanity.com/paper/2408.10908/) (2024)
- [Neuro-Cognitive Reward Modeling for Human-Centered Autonomous Vehicle Control](https://mlsanity.com/paper/2603.25968/) (2026)
- [AEGIS: Human Attention-based Explainable Guidance for Intelligent Vehicle Systems](https://mlsanity.com/paper/2504.05950/) (2025)
- [Causal-Entity Reflected Egocentric Traffic Accident Video Synthesis](https://mlsanity.com/paper/2506.23263/) (2025)
- [Estimating cognitive biases with attention-aware inverse planning](https://mlsanity.com/paper/2510.25951/) (2025)
- [From Scene to Object: Text-Guided Dual-Gaze Prediction](https://mlsanity.com/paper/2604.20191/) (2026)

## Abstract (arXiv)

We propose a computational model to estimate a person's attended awareness of their environment. We define attended awareness to be those parts of a potentially dynamic scene which a person has attended to in recent history and which they are still likely to be physically aware of. Our model takes as input scene information in the form of a video and noisy gaze estimates, and outputs visual saliency, a refined gaze estimate, and an estimate of the person's attended awareness. In order to test our model, we capture a new dataset with a high-precision gaze tracker including 24.5 hours of gaze sequences from 23 subjects attending to videos of driving scenes. The dataset also contains third-party annotations of the subjects' attended awareness based on observations of their scan path. Our results show that our model is able to reasonably estimate attended awareness in a controlled setting, and in the future could potentially be extended to real egocentric driving data to help enable more effective ahead-of-time warnings in safety systems and thereby augment driver performance. We also demonstrate our model's effectiveness on the tasks of saliency, gaze calibration, and denoising, using both our dataset and an existing saliency dataset. We make our model and dataset available at https://github.com/ToyotaResearchInstitute/att-aware/.