mlsanityData to 14 Oct 2026

EyeMulator: Improving Code Language Models by Mimicking Human Visual Attention

arXiv 2508.16771 ↗First posted 22 Aug 2025cs.SE cs.AI cs.HCcited by 4

Not on the map. Why?

Abstract (arXiv)

Code Language Models (CodeLLMs) learn token importance from data correlations, whereas human developers attend selectively to semantically salient code. We present EyeMulator, a model-agnostic method that injects human visual-attention priors into CodeLLM fine-tuning without architectural changes. EyeMulator distills eye-tracking data into semantic salience and gaze-transition priors, then uses them to reweight token-level training losses. Across six backbones, two data regimes, and three CodeXGLUE tasks, the reported configurations yield positive matched-metric deltas in all 36 model-task-setting cells. Effects are largest for structure-preserving completion and translation, while summarization shows smaller but positive METEOR deltas. Session-mode and component-ablation analyses further show that reading, writing, semantic, and transition-derived priors provide complementary signal. Human-attention artifacts are available at https://zenodo.org/records/17205682.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 39 of its 78 references that are in the map. First posted on arXiv 22 Aug 2025.

Builds on

39 of 78 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  8. 2017Deep reinforcement learning from human preferencescited by 3,912
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  10. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  11. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  12. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
27 more
  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 963
  2. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
  3. 2024A Survey on Large Language Models for Code Generationcited by 658
  4. 2024Executable Code Actions Elicit Better LLM Agentscited by 564
  5. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 479
  6. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 396
  7. 2023Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraintcited by 350
  8. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 264
  9. 2023Using an LLM to Help With Code Understandingcited by 228
  10. 2021Retrieval Augmented Code Generation and Summarizationcited by 149
  11. 2024LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigationcited by 93
  12. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 91
  13. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedbackcited by 77
  14. 2024A Performance Study of LLM-Generated Code on Leetcodecited by 49
  15. 2023Retrieval-Augmented Code Generation for Universal Information Extractioncited by 40
  16. 2023LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedbackcited by 39
  17. 2021Transitioning to human interaction with AI systems: New challenges and opportunities for HCI professionals to enable human-centered AIcited by 38
  18. 2024SemCoder: Training Code Language Models with Comprehensive Semantics Reasoningcited by 27
  19. 2024Do Machines and Humans Focus on Similar Code? Exploring Explainability of Large Language Models in Code Summarizationcited by 16
  20. 2025ELABORATION: A Comprehensive Benchmark on Human-LLM Competitive Programmingcited by 16
  21. 2024EyeTrans: Merging Human and Machine Attention for Neural Code Summarizationcited by 9
  22. 2023Towards Modeling Human Attention from Eye Movements for Neural Source Code Summarizationcited by 8
  23. 2023Modeling Programmer Attention as Scanpath Predictioncited by 7
  24. 2024An Empirical Study on Capability of Large Language Models in Understanding Code Semanticscited by 7
  25. 2025CodeGrad: Integrating Multi-Step Verification with Gradient-Based LLM Refinementcited by 7
  26. 2024Beyond Code Generation: Assessing Code LLM Maturity with Postconditionscited by 4
  27. 2023Where to Look When Repairing Code? Comparing the Attention of Neural Models and Developerscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025E2Edev: Benchmarking Large Language Models in End-to-End Software Development Taskcited by 6
  2. 2026EyeLayer: Integrating Human Attention Patterns into LLM-Based Code Summarizationcited by 0
  3. 2026Your Mouse and Eyes Secretly Leak Your Preference: LLM Alignment using Implicit Feedback from Userscited by 0
  4. 2026How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repaircited by 0