mlsanityData to 14 Oct 2026

OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses

arXiv 2503.10927 ↗First posted 13 Mar 2025cs.CL cs.AIcited by 3

On the map: LLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment

Abstract (arXiv)

While Large Language Models (LLMs) have significantly advanced natural language processing, aligning them with human preferences remains an open challenge. Although current alignment methods rely primarily on explicit feedback, eye-tracking (ET) data offers insights into real-time cognitive processing during reading. In this paper, we present OASST-ETC, a novel eye-tracking corpus capturing reading patterns from 24 participants, while evaluating LLM-generated responses from the OASST1 dataset. Our analysis reveals distinct reading patterns between preferred and non-preferred responses, which we compare with synthetic eye-tracking data. Furthermore, we examine the correlation between human reading measures and attention patterns from various transformer-based models, discovering stronger correlations in preferred responses. This work introduces a unique resource for studying human cognitive processing in LLM evaluation and suggests promising directions for incorporating eye-tracking data into alignment methods. The dataset and analysis code are publicly available.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 55 of its 108 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

55 of 108 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedback2 Oct
  2. 2023Synthesizing Human Gaze Feedback for Improved NLP Performance11 Feb
  3. 2022Constitutional AI: Harmlessness from AI Feedback15 Dec

Other references · 52

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  8. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  9. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  10. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  11. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
  12. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 499
38 more
  1. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 463
  2. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Trainingcited by 401
  3. 2023AI Alignment: A Comprehensive Surveycited by 263
  4. 2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 236
  5. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 204
  6. 2024Advancing LLM Reasoning Generalists with Preference Treescited by 192
  7. 2023HelpSteer: Multi-attribute Helpfulness Dataset for SteerLMcited by 142
  8. 2023SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHFcited by 111
  9. 2019ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotationcited by 66
  10. 2024Quantile Regression for Distributional Reward Models in RLHFcited by 60
  11. 2023Pairwise Proximal Policy Optimization: Harnessing Relative Feedback for LLM Alignmentcited by 57
  12. 2020Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attentioncited by 45
  13. 2023RLCD: Reinforcement Learning from Contrastive Distillation for Language Model Alignmentcited by 43
  14. 2020Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehensioncited by 29
  15. 2021Multilingual Language Models Predict Human Reading Behaviorcited by 29
  16. 2019Entity Recognition at First Sight: Improving NER with Eye Movement Informationcited by 26
  17. 2021Relative Importance in Sentence Processingcited by 22
  18. 2022Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?cited by 20
  19. 2017Leveraging Cognitive Features for Sentiment Analysiscited by 19
  20. 2023Eyettention: An Attention-based Dual-Sequence Model for Predicting Human Scanpaths during Readingcited by 19
  21. 2019Advancing NLP with Cognitive Language Processing Signalscited by 18
  22. 2024HRLAIF: Improvements in Helpfulness and Harmlessness in Open-domain Reinforcement Learning From AI Feedbackcited by 15
  23. 2023ScanDL: A Diffusion Model for Generating Synthetic Scanpaths on Textscited by 14
  24. 2024Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Modelscited by 12
  25. 2021CogAlign: Learning to Align Textual Neural Representations to Cognitive Language Processing Signalscited by 11
  26. 2021A Survey on Using Gaze Behaviour for Natural Language Processingcited by 11
  27. 2018Eyes are the Windows to the Soul: Predicting the Rating of Text Quality Using Gaze Behaviourcited by 9
  28. 2020Happy Are Those Who Grade without Seeing: A Multi-Task Learning Approach to Grade Essays Using Gaze Behaviourcited by 9
  29. 2024EyeTrans: Merging Human and Machine Attention for Neural Code Summarizationcited by 9
  30. 2023Pre-Trained Language Models Augmented with Synthetic Scanpaths for Natural Language Understandingcited by 8
  31. 2021TorontoCL at CMCL 2021 Shared Task: RoBERTa with Multi-Stage Fine-Tuning for Eye-Tracking Predictioncited by 7
  32. 2022The Copenhagen Corpus of Eye Tracking Recordings from Natural Reading of Danish Textscited by 7
  33. 2023UniAR: A Unified model for predicting human Attention and Responses on visual contentcited by 7
  34. 2023WebQAmGaze: A Multilingual Webcam Eye-Tracking-While-Reading Datasetcited by 6
  35. 2023Probing Large Language Models from A Human Behavioral Perspectivecited by 5
  36. 2024PoTeC: A German Naturalistic Eye-tracking-while-reading Corpuscited by 5
  37. 2024EMTeC: A Corpus of Eye Movements on Machine-Generated Textscited by 5
  38. 2024Reporting Eye-Tracking Data Quality: Towards a New Standardcited by 4

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Gaze patterns predict preference and confidence in pairwise AI image evaluationcited by 1
  2. 2026Eye-Tracking-while-Reading: A Living Survey of Datasets with Open Library Supportcited by 0
  3. 2026Your Mouse and Eyes Secretly Leak Your Preference: LLM Alignment using Implicit Feedback from Userscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Integrating Cognitive Processing Signals into Language Models: A Review of Advances, Applications and Future Directions
  2. 2026Do VLMs Align Better with Humans than LLMs during Natural Reading?
  3. 2024Fine-Grained Prediction of Reading Comprehension from Eye Movements
  4. 2025Psycholinguistic Word Features: a New Approach for the Evaluation of LLMs Alignment with Humans
  5. 2025Large Language Models Are Human-Like Internally
  6. 2025Uncovering the Computational Ingredients of Human-Like Representations in LLMs
  7. 2023Instruction-tuning Aligns LLMs to the Human Brain
  8. 2023Large Language Models as Automated Aligners for benchmarking Vision-Language Models