# DiffGaze: A Diffusion Model for Continuous Gaze Sequence Generation on 360deg Images

arXiv: https://arxiv.org/abs/2403.17477 · HTML: https://mlsanity.com/paper/2403.17477/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 15 of its 49 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation](https://mlsanity.com/paper/2107.03502/) (2021)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning](https://mlsanity.com/paper/2005.14310/) (2020)
- [CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research](https://mlsanity.com/paper/1505.03581/) (2015)
- [Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-based Interaction](https://mlsanity.com/paper/1405.0006/) (2014)
- [SaltiNet: Scan-path Prediction on 360 Degree Images using Saliency Volumes](https://mlsanity.com/paper/1707.03123/) (2017)
- [AiR: Attention with Reasoning Capability](https://mlsanity.com/paper/2007.14419/) (2020)
- [ScanGAN360: A Generative Model of Realistic Scanpaths for 360circ Images](https://mlsanity.com/paper/2103.13922/) (2021)
- [SalNet360: Saliency Maps for omni-directional images with CNN](https://mlsanity.com/paper/1709.06505/) (2017)
- [Scanpath Prediction on Information Visualisations](https://mlsanity.com/paper/2112.02340/) (2021)

## Built on by (most cited first, 2 of 2)

- [LAP: A Language-Aware Planning Model For Procedure Planning In Instructional Videos](https://mlsanity.com/paper/2603.09743/) (2026)
- [Beyond Scanpaths: Graph-Based Gaze Simulation in Dynamic Scenes](https://mlsanity.com/paper/2603.28319/) (2026)

## Abstract (arXiv)

We present DiffGaze, a novel method for generating realistic and diverse continuous human gaze sequences on 360deg images based on a conditional score-based denoising diffusion model. Generating human gaze on 360deg images is important for various human-computer interaction and computer graphics applications, e.g. for creating large-scale eye tracking datasets or for realistic animation of virtual humans. However, existing methods are limited to predicting discrete fixation sequences or aggregated saliency maps, thereby neglecting crucial parts of natural gaze behaviour. Our method uses features extracted from 360deg images as condition and uses two transformers to model the temporal and spatial dependencies of continuous human gaze. We evaluate DiffGaze on two 360deg image benchmarks for gaze sequence generation as well as scanpath prediction and saliency prediction. Our evaluations show that DiffGaze outperforms state-of-the-art methods on all tasks on both benchmarks. We also report a 21-participant user study showing that our method generates gaze sequences that are indistinguishable from real human sequences.