mlsanityData to 14 Oct 2026

Enhancing RLHF with Human Gaze Modeling

arXiv 2507.09016 ↗First posted 11 Jul 2025cs.LGcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

Reinforcement Learning from Human Feedback (RLHF) aligns language models with human preferences but is computationally expensive. We explore two approaches that leverage human gaze modeling to enhance RLHF: (1) gaze-aware reward models and (2) gaze-based distribution of sparse rewards at token level. Our experiments demonstate that gaze-informed RLHF achieves faster convergence while maintaining or slightly improving performance, thus, reducing computational costs during policy optimization. These results show that human gaze provides a valuable and underused signal for policy optimization, pointing to a promising direction for improving RLHF efficiency.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 6 of its 15 references that are in the map. First posted on arXiv 11 Jul 2025.

Builds on

6 of 15 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Dense Reward for Free in Reinforcement Learning from Human Feedback1 Feb
  2. 2019Fine-Tuning Language Models from Human Preferences18 Sep

Other references · 4

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  3. 2023Identifying and Mitigating Vulnerabilities in LLM-Integrated Applicationscited by 54
  4. 2024Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Modelscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Gaze patterns predict preference and confidence in pairwise AI image evaluationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs
  2. 2025A Unified Pairwise Framework for RLHF: Bridging Generative Reward Modeling and Policy Optimization
  3. 2023A Survey of Reinforcement Learning from Human Feedback
  4. 2026Reinforcement Learning from Human Feedback: A Statistical Perspective
  5. 2023The History and Risks of Reinforcement Learning and Human Feedback
  6. 2024Secrets of RLHF in Large Language Models Part II: Reward Modeling
  7. 2024RLHF Workflow: From Reward Modeling to Online RLHF
  8. 2025Provably Efficient Online RLHF with One-Pass Reward Modeling