mlsanityData to 7 Oct 2026

Towards Modeling Human Attention from Eye Movements for Neural Source Code Summarization

arXiv 2305.09773 ↗First posted 16 May 2023cs.SE cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Neural source code summarization is the task of generating natural language descriptions of source code behavior using neural networks. A fundamental component of most neural models is an attention mechanism. The attention mechanism learns to connect features in source code to specific words to use when generating natural language descriptions. Humans also pay attention to some features in code more than others. This human attention reflects experience and high-level cognition well beyond the capability of any current neural model. In this paper, we use data from published eye-tracking experiments to create a model of this human attention. The model predicts which words in source code are the most important for code summarization. Next, we augment a baseline neural code summarization approach using our model of human attention. We observe an improvement in prediction performance of the augmented approach in line with other bio-inspired neural models.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 19 of its 63 references that are in the map. First posted on arXiv 16 May 2023.

Builds on

19 of 63 references.

References · 19

  1. 2017Learning to Represent Programs with Graphscited by 219
  2. 2018The Adverse Effects of Code Duplication in Machine Learning Models of Codecited by 123
  3. 2017Building machines that adapt and compute like brainscited by 99
  4. 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
  5. 2017Shake-Shake regularizationcited by 87
  6. 2020Improved Code Summarization via a Graph Neural Networkcited by 79
  7. 2019A Neural Model for Generating Natural Language Summaries of Program Subroutinescited by 76
  8. 2021Language-Agnostic Representation Learning of Source Code from Structure and Contextcited by 63
  9. 2018Decoding Brain Representations by Multimodal Learning of Neural Activity and Visual Featurescited by 52
  10. 2019Recommendations for Datasets for Source Code Summarizationcited by 46
  11. 2020Improved Automatic Summarization of Subroutines via Attention to File Contextcited by 34
  12. 2021Project-Level Encoding for Neural Source Code Summarization of Subroutinescited by 24
7 more
  1. 2016Top-down Visual Saliency Guided by Captionscited by 22
  2. 2017Using Human Brain Activity to Guide Machine Learningcited by 15
  3. 2022Semantic Similarity Metrics for Evaluating Source Code Summarizationcited by 14
  4. 2021Action Word Prediction for Neural Source Code Summarizationcited by 11
  5. 2017Building Machines that Learn and Think for Themselves: Commentary on Lake et al., Behavioral and Brain Sciences, 2017cited by 5
  6. 2019Developer Reading Behavior While Summarizing Java Methods: Size and Context Matterscited by 3
  7. 2021gazel: Supporting Source Code Edits in Eye-Tracking Studiescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 1; 2025: 2; 2026: 2.

8 papers.

  1. 2023Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?cited by 13
  2. 2024EyeTrans: Merging Human and Machine Attention for Neural Code Summarizationcited by 9
  3. 2023Roles of Scaling and Instruction Tuning in Language Perception: Model vs. Human Attentioncited by 7
  4. 2025EyeMulator: Improving Code Language Models by Mimicking Human Visual Attentioncited by 4
  5. 2023Revisiting File Context for Source Code Summarizationcited by 3
  6. 2025Enhancing Code LLM Training with Programmer Attentioncited by 0
  7. 2026EyeLayer: Integrating Human Attention Patterns into LLM-Based Code Summarizationcited by 0
  8. 2026Synthetic Semantic Supervision for Contrastive Code Representation Learning in Small Transformers: An Empirical Studycited by 0