mlsanityData to 7 Oct 2026

Gaze patterns predict preference and confidence in pairwise AI image evaluation

arXiv 2603.24849 ↗First posted 25 Mar 2026cs.HC cs.AI cs.CVcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

Preference learning methods, such as Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO), rely on pairwise human judgments, yet little is known about the cognitive processes underlying these judgments. We investigate whether eye-tracking can reveal preference formation during pairwise AI-generated image evaluation. Thirty participants completed 1,800 trials while their gaze was recorded. We replicated the gaze cascade effect, with gaze shifting toward chosen images approximately one second before the decision. Cascade dynamics were consistent across confidence levels. Gaze features predicted binary choice (68% accuracy), with chosen images receiving more dwell time, fixations, and revisits. Gaze transitions distinguished high-confidence from uncertain decisions (66% accuracy), with low-confidence trials showing more image switches per second. These results show that gaze patterns predict both choice and confidence in pairwise image evaluations, suggesting that eye-tracking provides implicit signals relevant to the quality of preference annotations.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 21 of its 39 references that are in the map. First posted on arXiv 25 Mar 2026.

Builds on

21 of 39 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Enhancing RLHF with Human Gaze Modeling11 Jul
  2. 2024Secrets of RLHF in Large Language Models Part II: Reward Modeling11 Jan
  3. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback27 Jul
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May

Other references · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  3. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  4. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  5. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  6. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  7. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  8. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
  9. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
  10. 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
  11. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
  12. 2023Rich Human Feedback for Text-to-Image Generationcited by 139
5 more
  1. 2024Learning Multi-dimensional Human Preference for Text-to-Image Generationcited by 102
  2. 2024Geometric-Averaged Preference Optimization for Soft Preference Labelscited by 18
  3. 2024Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Modelscited by 11
  4. 2024Finding the Subjective Truth: Collecting 2 Million Votes for Comprehensive Gen-AI Model Evaluationcited by 7
  5. 2025OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responsescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Your Mouse and Eyes Secretly Leak Your Preference: LLM Alignment using Implicit Feedback from Userscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Similar Choices, Different Attention: Cross-Modal Associations in Humans and Vision-Language Models
  2. 2026Aligning to Illusions: Choice Blindness in Human and AI Feedback
  3. 2026Visual Persuasion: What Influences Decisions of Vision-Language Models?
  4. 2025Synthesizing Images on Perceptual Boundaries of ANNs for Uncovering and Manipulating Human Perceptual Variability
  5. 2025Learning a Canonical Basis of Human Preferences from Binary Ratings
  6. 2024Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Survey
  7. 2026RLHF May Not Reflect Genuine Preferences
  8. 2025Visual Categorization Across Minds and Models: Cognitive Analysis of Human Labeling and Neuro-Symbolic Integration