# ScanGAN360: A Generative Model of Realistic Scanpaths for 360circ Images

arXiv: https://arxiv.org/abs/2103.13922 · HTML: https://mlsanity.com/paper/2103.13922/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 80 references that are in the map.
- First posted on arXiv 25 Mar 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017)
- [Predicting Human Eye Fixations via an LSTM-based Saliency Attentive Model](https://mlsanity.com/paper/1611.09571/) (2016)
- [D3TW: Discriminative Differentiable Dynamic Time Warping for Weakly Supervised Action Alignment and Segmentation](https://mlsanity.com/paper/1901.02598/) (2019)
- [Deep Visual Attention Prediction](https://mlsanity.com/paper/1705.02544/) (2017)
- [Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models](https://mlsanity.com/paper/1909.09020/) (2019)
- [SalGAN: Visual Saliency Prediction with Generative Adversarial Networks](https://mlsanity.com/paper/1701.01081/) (2017)
- [DeepGaze II: Reading fixations from deep features trained on object recognition](https://mlsanity.com/paper/1610.01563/) (2016)
- [Shallow and Deep Convolutional Networks for Saliency Prediction](https://mlsanity.com/paper/1603.00845/) (2016)
- [Predicting Head Movement in Panoramic Video: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1710.10755/) (2017)
- [PathGAN: Visual Scanpath Prediction with Generative Adversarial Networks](https://mlsanity.com/paper/1809.00567/) (2018)
- [SaltiNet: Scan-path Prediction on 360 Degree Images using Saliency Volumes](https://mlsanity.com/paper/1707.03123/) (2017)
- [Differentiable Divergences Between Time Series](https://mlsanity.com/paper/2010.08354/) (2020)
- [Pano2Vid: Automatic Cinematography for Watching 360circ Videos](https://mlsanity.com/paper/1612.02335/) (2016)
- [State-of-the-Art in Human Scanpath Prediction](https://mlsanity.com/paper/2102.12239/) (2021)
- [SalNet360: Saliency Maps for omni-directional images with CNN](https://mlsanity.com/paper/1709.06505/) (2017)
- [Making 360circ Video Watchable in 2D: Learning Videography for Click Free Viewing](https://mlsanity.com/paper/1703.00495/) (2017)
- [GazeGAN - Unpaired Adversarial Image Generation for Gaze Estimation](https://mlsanity.com/paper/1711.09767/) (2017)
- [Very Long Term Field of View Prediction for 360-degree Video Streaming](https://mlsanity.com/paper/1902.01439/) (2019)
- [Multimodality in VR: A survey](https://mlsanity.com/paper/2101.07906/) (2021)
- [Movie Editing and Cognitive Event Segmentation in Virtual Reality Video](https://mlsanity.com/paper/1806.04924/) (2018)
- [Prediction of the Influence of Navigation Scan-path on Perceived Quality of Free-Viewpoint Videos](https://mlsanity.com/paper/1810.04409/) (2018)

## Built on by (most cited first, 23 of 23)

- [Elite360D: Towards Efficient 360 Depth Estimation via Semantic- and Distance-Aware Bi-Projection Fusion](https://mlsanity.com/paper/2403.16376/) (2024)
- [Assessor360: Multi-sequence Network for Blind Omnidirectional Image Quality Assessment](https://mlsanity.com/paper/2305.10983/) (2023)
- [Synthesizing Human Gaze Feedback for Improved NLP Performance](https://mlsanity.com/paper/2302.05721/) (2023)
- [Eye-tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges](https://mlsanity.com/paper/2305.14080/) (2023)
- [3D VR Sketch Guided 3D Shape Prototyping and Exploration](https://mlsanity.com/paper/2306.10830/) (2023)
- [PANORAMA: The Rise of Omnidirectional Vision in the Embodied AI Era](https://mlsanity.com/paper/2509.12989/) (2025)
- [An Overview on Generative AI at Scale with Edge-Cloud Computing](https://mlsanity.com/paper/2306.17170/) (2023)
- [Dream360: Diverse and Immersive Outdoor Virtual Scene Creation via Transformer-Based 360 Image Outpainting](https://mlsanity.com/paper/2401.10564/) (2024)
- [EyeFormer: Predicting Personalized Scanpaths with Transformer-Guided Reinforcement Learning](https://mlsanity.com/paper/2404.10163/) (2024)
- [Perceptual Quality Assessment of 360^circ Images Based on Generative Scanpath Representation](https://mlsanity.com/paper/2309.03472/) (2023)
- [DiffEye: Diffusion-Based Continuous Eye-Tracking Data Generation Conditioned on Natural Images](https://mlsanity.com/paper/2509.16767/) (2025)
- [Scanpath Prediction in Panoramic Videos via Expected Code Length Minimization](https://mlsanity.com/paper/2305.02536/) (2023)
- [GazeFusion: Saliency-Guided Image Generation](https://mlsanity.com/paper/2407.04191/) (2024)
- [TPP-Gaze: Modelling Gaze Dynamics in Space and Time with Neural Temporal Point Processes](https://mlsanity.com/paper/2410.23409/) (2024)
- [DiffGaze: A Diffusion Model for Continuous Gaze Sequence Generation on 360deg Images](https://mlsanity.com/paper/2403.17477/) (2024)
- [Impact of Design Decisions in Scanpath Modeling](https://mlsanity.com/paper/2405.08981/) (2024)
- [Pathformer3D: A 3D Scanpath Transformer for 360deg Images](https://mlsanity.com/paper/2407.10563/) (2024)
- [Embedding-Driven Data Distillation for 360-Degree IQA With Residual-Aware Refinement](https://mlsanity.com/paper/2412.12667/) (2024)
- [Generating Realistic Synthetic Head Rotation Data for Extended Reality using Deep Learning](https://mlsanity.com/paper/2501.09050/) (2025)
- [A Comparative Study of Scanpath Models in Graph-Based Visualization](https://mlsanity.com/paper/2503.24160/) (2025)
- [Camera-based implicit mind reading by capturing higher-order semantic dynamics of human gaze within environmental context](https://mlsanity.com/paper/2507.12889/) (2025)
- [Infinite Gaze Generation for Videos with Autoregressive Diffusion](https://mlsanity.com/paper/2603.24938/) (2026)
- [Object Referring-Guided Scanpath Prediction with Perception-Enhanced Vision-Language Models](https://mlsanity.com/paper/2604.20361/) (2026)

## Abstract (arXiv)

Understanding and modeling the dynamics of human gaze behavior in 360^circ environments is a key challenge in computer vision and virtual reality. Generative adversarial approaches could alleviate this challenge by generating a large number of possible scanpaths for unseen images. Existing methods for scanpath generation, however, do not adequately predict realistic scanpaths for 360^circ images. We present ScanGAN360, a new generative adversarial approach to address this challenging problem. Our network generator is tailored to the specifics of 360^circ images representing immersive environments. Specifically, we accomplish this by leveraging the use of a spherical adaptation of dynamic-time warping as a loss function and proposing a novel parameterization of 360^circ scanpaths. The quality of our scanpaths outperforms competing approaches by a large margin and is almost on par with the human baseline. ScanGAN360 thus allows fast simulation of large numbers of virtual observers, whose behavior mimics real users, enabling a better understanding of gaze behavior and novel applications in virtual scene design.