# GazeFlow: From Human Gaze Behavior to Generative Egocentric Gaze Prediction

arXiv: https://arxiv.org/abs/2609.38519 · HTML: https://mlsanity.com/paper/2609.38519/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 54 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Improving and generalizing flow-based generative models with minibatch optimal transport](https://mlsanity.com/paper/2302.00482/) (2023)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022)
- [What do different evaluation metrics tell us about saliency models?](https://mlsanity.com/paper/1604.03605/) (2016)
- [Predicting the Driver's Focus of Attention: the DR(eye)VE Project](https://mlsanity.com/paper/1705.03854/) (2017)
- [Predicting Gaze in Egocentric Video by Learning Task-dependent Attention Transition](https://mlsanity.com/paper/1803.09125/) (2018)
- [In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimation](https://mlsanity.com/paper/2208.04464/) (2022)
- [MetaSapiens: Real-Time Neural Rendering with Efficiency-Aware Pruning and Accelerated Foveated Rendering](https://mlsanity.com/paper/2407.00435/) (2024)
- [Listen to Look into the Future: Audio-Visual Egocentric Gaze Anticipation](https://mlsanity.com/paper/2305.03907/) (2023)
- [Gaze-based dual resolution deep imitation learning for high-precision dexterous robot manipulation](https://mlsanity.com/paper/2102.01295/) (2021)
- [EgoM2P: Egocentric Multimodal Multitask Pretraining](https://mlsanity.com/paper/2506.07886/) (2025)
- [Color-Perception-Guided Display Power Reduction for Virtual Reality](https://mlsanity.com/paper/2209.07610/) (2022)
- [DiffEye: Diffusion-Based Continuous Eye-Tracking Data Generation Conditioned on Natural Images](https://mlsanity.com/paper/2509.16767/) (2025)
- [LowPowAR: Power-Constrained Tone Mapping for Augmented Reality](https://mlsanity.com/paper/2607.19509/) (2026)
- [Vignette: Perceptual Compression for Video Storage and Processing Systems](https://mlsanity.com/paper/1902.01372/) (2019)
- [PowerGS: Display-Rendering Power Co-Optimization for Neural Rendering in Power-Constrained XR Systems](https://mlsanity.com/paper/2509.21702/) (2025)
- [Modeling and Exploiting the Time Course of Chromatic Adaptation for Display Power Optimizations in Virtual Reality](https://mlsanity.com/paper/2509.23489/) (2025)

## Abstract (arXiv)

Egocentric gaze prediction enables many downstream applications but remains challenging, as human gaze is inherently stochastic. This stochasticity is constrained by structured temporal dynamics alternating between fixations and saccades, top-down influences from tasks, and bottom-up visual saliency. Based on this observation, we introduce GazeFlow, a framework that directly models gaze as a joint distribution of temporal gaze positions conditioned upon both top-down and bottom-up information. In particular, GazeFlow uses conditional flow matching (CFM): a learned velocity field iteratively transports a Gaussian noise sample into a plausible gaze trajectory drawn from this joint distribution. The velocity field is conditioned on bottom-up visual features extracted by a video encoder and on top-down task information obtained by globally querying these features. On standard datasets, GazeFlow achieves state-of-the-art performance on per-frame metrics, and the generated trajectories align better with human gaze temporal dynamics.