# TransGaze-Object: Transformer Based Driver Gaze Object Prediction Framework in Real Driving

arXiv: https://arxiv.org/abs/2609.10139 · HTML: https://mlsanity.com/paper/2609.10139/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 57 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [IDD: A Dataset for Exploring Problems of Autonomous Navigation in Unconstrained Environments](https://mlsanity.com/paper/1811.10200/) (2018)
- [Predicting the Driver's Focus of Attention: the DR(eye)VE Project](https://mlsanity.com/paper/1705.03854/) (2017)
- [Gaze Estimation using Transformer](https://mlsanity.com/paper/2105.14424/) (2021)
- [DMD: A Large-Scale Multi-Modal Driver Monitoring Dataset for Attention and Alertness Analysis](https://mlsanity.com/paper/2008.12085/) (2020)
- [Object-aware Gaze Target Detection](https://mlsanity.com/paper/2307.09662/) (2023)
- [Driver Gaze Zone Estimation using Convolutional Neural Networks: A General Framework and Ablative Analysis](https://mlsanity.com/paper/1802.02690/) (2018)
- [Speak2Label: Using Domain Knowledge for Creating a Large Scale Driver Gaze Zone Estimation Dataset](https://mlsanity.com/paper/2004.05973/) (2020)
- [Dynamics of Driver's Gaze: Explorations in Behavior Modeling & Maneuver Prediction](https://mlsanity.com/paper/1802.00066/) (2018)
- [What Do You See in Vehicle? Comprehensive Vision Solution for In-Vehicle Gaze Estimation](https://mlsanity.com/paper/2403.15664/) (2024)
- [A High-Level Description and Performance Evaluation of Pupil Invisible](https://mlsanity.com/paper/2009.00508/) (2020)
- [A Review of Driver Gaze Estimation and Application in Gaze Behavior Understanding](https://mlsanity.com/paper/2307.01470/) (2023)
- [Owl and Lizard: Patterns of Head Pose and Eye Pose in Driver Gaze Classification](https://mlsanity.com/paper/1508.04028/) (2015)
- [GazeVLM: A Vision-Language Model for Multi-Task Gaze Understanding](https://mlsanity.com/paper/2511.06348/) (2025)
- [TransGOP: Transformer-Based Gaze Object Prediction](https://mlsanity.com/paper/2402.13578/) (2024)
- [EraW-Net: Enhance-Refine-Align W-Net for Scene-Associated Driver Attention Estimation](https://mlsanity.com/paper/2408.08570/) (2024)
- [Cross-Paradigm Evaluation of Gaze-Based Semantic Object Identification for Intelligent Vehicles](https://mlsanity.com/paper/2602.01452/) (2026)
- [SGAP-Gaze: Scene Grid Attention Based Point-of-Gaze Estimation Network for Driver Gaze](https://mlsanity.com/paper/2604.19888/) (2026)

## Abstract (arXiv)

Driver gaze provides information regarding driver visual attention and situational awareness to the surrounding traffic. Existing driver gaze estimation studies represent gaze in terms of gaze zone or gaze vector/point-of-gaze (PoG). However, object-level gaze information provides a more semantically meaningful representation of visual attention by identifying attended objects, such as vehicles, pedestrians, or traffic signals. In this study, we propose an end-to-end driver gaze object prediction framework, TransGaze-Object, Transformer-based Gaze Object prediction model. The proposed framework first extracts facial features, including face and iris-weighted eye features, along with trafficobject spatial features. A transformer based cross-attention mechanism is then used to compute similarity scores and attention weights for predicting the drivers gaze object. To train this model, we propose a benchmark driver gaze dataset, Urban Driving-Face Scene Gaze (UD-FSG), comprising synchronized driver-face and traffic-scene images, scene objects bounding boxes, and gaze labels in terms of 2D gaze coordinate and gaze object. The TransGaze-Object model achieves an overall accuracy of 60% for gaze-object prediction, compared to 51% accuracy obtained from associating the estimated Point-of-Gaze to traffic objects. The error analysis reveals that TransGaze-Object reduces confusion between traffic objects (predicted) and the background (ground-truth), achieving an error rate of 11.68%, a 49.7% relative reduction compared with 23.21% error obtained from PoG-based gaze-object association. Overall, the results demonstrate the effectiveness of directly predicting gaze objects from driver-face and traffic-scene information, rather than estimating an intermediate Point-of-Gaze and subsequently associating it with traffic objects.