# Enhancing RLHF with Human Gaze Modeling

arXiv: https://arxiv.org/abs/2507.09016 · HTML: https://mlsanity.com/paper/2507.09016/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 6 of its 15 references that are in the map.
- First posted on arXiv 11 Jul 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Reward modeling and RLHF](https://mlsanity.com/alignment/reward-modeling-and-rlhf/)

## Builds on (most cited first, 6 of 6)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [Dense Reward for Free in Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2402.00782/) (2024)
- [Identifying and Mitigating Vulnerabilities in LLM-Integrated Applications](https://mlsanity.com/paper/2311.16153/) (2023)
- [Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models](https://mlsanity.com/paper/2410.01532/) (2024)

## Built on by (most cited first, 1 of 1)

- [Gaze patterns predict preference and confidence in pairwise AI image evaluation](https://mlsanity.com/paper/2603.24849/) (2026)

## Abstract (arXiv)

Reinforcement Learning from Human Feedback (RLHF) aligns language models with human preferences but is computationally expensive. We explore two approaches that leverage human gaze modeling to enhance RLHF: (1) gaze-aware reward models and (2) gaze-based distribution of sparse rewards at token level. Our experiments demonstate that gaze-informed RLHF achieves faster convergence while maintaining or slightly improving performance, thus, reducing computational costs during policy optimization. These results show that human gaze provides a valuable and underused signal for policy optimization, pointing to a promising direction for improving RLHF efficiency.