mlsanityData to 14 Oct 2026

Beyond Scanpaths: Graph-Based Gaze Simulation in Dynamic Scenes

arXiv 2603.28319 ↗First posted 30 Mar 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Accurately modelling human attention is essential for numerous computer vision applications, particularly in the domain of automotive safety. Existing methods typically collapse gaze into saliency maps or scanpaths, treating gaze dynamics only implicitly. We instead formulate gaze modelling as an autoregressive dynamical system and explicitly unroll raw gaze trajectories over time, conditioned on both gaze history and the evolving environment. Driving scenes are represented as gaze-centric graphs processed by the Affinity Relation Transformer (ART), a heterogeneous graph transformer that models interactions between driver gaze, traffic objects, and road structure. We further introduce the Object Density Network (ODN) to predict next-step gaze distributions, capturing the stochastic and object-centric nature of attentional shifts in complex environments. We also release Focus100, a new dataset of raw gaze data from 30 participants viewing egocentric driving footage. Trained directly on raw gaze, without fixation filtering, our unified approach produces more natural gaze trajectories, scanpath dynamics, and saliency maps than existing attention models, offering valuable insights for the temporal modelling of human attention in dynamic environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 149 references that are in the map. First posted on arXiv 30 Mar 2026.

Builds on

15 of 149 references.

References · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2017Graph Attention Networkscited by 8,923
  4. 2017The Kinetics Human Action Video Datasetcited by 1,909
  5. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,167
  6. 2018Self-Attention with Relative Position Representationscited by 1,120
  7. 2020Learning Mesh-Based Simulation with Graph Networkscited by 868
  8. 2017SalGAN: Visual Saliency Prediction with Generative Adversarial Networkscited by 65
  9. 2014Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNetcited by 60
  10. 2023Eagle: Large-Scale Learning of Turbulent Fluid Dynamics with Mesh Transformerscited by 57
  11. 2022YOLOPv2: Better, Faster, Stronger for Panoptic Driving Perceptioncited by 26
  12. 2021Heterogeneous Edge-Enhanced Graph Attention Network For Multi-Agent Trajectory Predictioncited by 17
3 more
  1. 2024Data Limitations for Modeling Top-Down Effects on Drivers' Attentioncited by 3
  2. 2024Learning Physical Simulation with Message Passing Transformercited by 3
  3. 2024DiffGaze: A Diffusion Model for Continuous Gaze Sequence Generation on 360deg Imagescited by 2

Built on by

Not cited yet.