TextGaze: Prompting Gaze Target Estimation with Textual Scene Cues
Not on the map. Why?
Abstract (arXiv)
Gaze target estimation aims to infer the position of a person's gaze within a scene. Within mainstream design logic, multi-branch methods require extra supervision and annotations, while streamlined designs prioritize low-level visual saliency over true gaze intent. The former leads to a high annotation burden and hinders domain transfer, whereas the latter causes misalignment between predicted attention and actual gaze targets. To address this issue, we propose TextGaze, a unified cross-modal architecture that leverages a Large Vision-Language Model (LVLM) as scalable semantic guidance to balance the two design paradigms. The model extracts visual features from a frozen encoder and utilizes an LVLM to obtain gaze-aligned textual cues. We design a transformer-based fusion module with hierarchical text supervision to preserve task semantics. Lightweight decoding heads enable the joint prediction of gaze heatmaps and in-/out-of-frame status. We evaluate our method on four mainstream datasets, and the results show competitive performance across key metrics with robust cross-dataset generalisation without extra fine-tuning. Overall, we provide a streamlined alternative to traditional designs and highlight the potential of LVLMs as accessible auxiliary guidance for gaze estimation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 52 references that are in the map. First posted on arXiv 11 Jul 2026.
Builds on
31 of 52 references.
References · 31
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
- 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
- 2020Detecting Attended Visual Targets in Videocited by 77
- 2018Connecting Gaze, Scene, and Attention: Generalized Attention Estimation via Joint Modeling of Gaze and Scene Saliencycited by 53
19 more
- 2019Believe It or Not, We Know What You Are Looking at!cited by 38
- 2024Gaze-LLE: Gaze Target Estimation via Large-Scale Learned Encoderscited by 36
- 2023ChildPlay: A New Benchmark for Understanding Children's Gaze Behaviourcited by 35
- 2024Unveiling Visual Perception in Language Models: An Attention Head Analysis Approachcited by 35
- 2023A Modular Multimodal Architecture for Gaze Target Prediction: Application to Privacy-Sensitive Settingscited by 31
- 2024Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecturecited by 28
- 2023EulerMormer: Robust Eulerian Motion Magnification via Dynamic Filtering within Transformercited by 27
- 2021GOO: A Dataset for Gaze Object Prediction in Retail Environmentscited by 26
- 2022Multimodal Across Domains Gaze Target Detectioncited by 22
- 2022Patch-level Gaze Distribution Prediction for Gaze Followingcited by 22
- 2023ViGT: Proposal-free Video Grounding with Learnable Token in Transformercited by 22
- 2024Prototypical Calibrating Ambiguous Samples for Micro-Action Recognitioncited by 22
- 2023Data Augmentation for Human Behavior Analysis in Multi-Person Conversationscited by 15
- 2024Exploring the Zero-Shot Capabilities of Vision-Language Models for Improving Gaze Followingcited by 14
- 2024AI Governance and Accountability: An Analysis of Anthropic's Claudecited by 14
- 2026XInsight: Integrative Stage-Consistent Psychological Counseling Support Agents for Digital Well-Beingcited by 9
- 2026MA-Bench: Towards Fine-grained Micro-Action Understandingcited by 9
- 2024GazeHTA: End-to-end Gaze Target Detection with Head-Target Associationcited by 6
- 2025Towards Seamless Interaction: Causal Turn-Level Modeling of Interactive 3D Conversational Head Dynamicscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Multi-scale Object-Aware Gaze Estimation via Geometric Reasoning
- 2024LG-Gaze: Learning Geometry-aware Continuous Prompts for Language-Guided Gaze Estimation
- 2024CLIP-Gaze: Towards General Gaze Estimation via Visual-Linguistic Model
- 2026PAGE: Towards Practical Human-level Gaze Target Estimation
- 2025GazeVLM: A Vision-Language Model for Multi-Task Gaze Understanding
- 2025VL4Gaze: Unleashing Vision-Language Models for Gaze Following
- 2026GazeMoE: Perception of Gaze Target with Mixture-of-Experts
- 2026Gaze-to-text Generation: Beyond Categorical Decoding of Human Attention