mlsanityData to 7 Oct 2026

Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?

arXiv 2306.01220 ↗First posted 2 Jun 2023cs.SE cs.HC cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have recently been widely used for code generation. Due to the complexity and opacity of LLMs, little is known about how these models generate code. We made the first attempt to bridge this knowledge gap by investigating whether LLMs attend to the same parts of a task description as human programmers during code generation. An analysis of six LLMs, including GPT-4, on two popular code generation benchmarks revealed a consistent misalignment between LLMs' and programmers' attention. We manually analyzed 211 incorrect code snippets and found five attention patterns that can be used to explain many code generation errors. Finally, a user study showed that model attention computed by a perturbation-based method is often favored by human programmers. Our findings highlight the need for human-aligned LLMs for better interpretability and programmer trust.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 67 of its 105 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

67 of 105 references.

References · 67

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  9. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  10. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  11. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  12. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
38 more
  1. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  2. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  3. 2021Measuring Coding Challenge Competence With APPScited by 879
  4. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  5. 2020Aligning AI With Shared Human Valuescited by 706
  6. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  7. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  8. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  9. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
  10. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  11. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  12. 2022CodeT: Code Generation with Generated Testscited by 369
  13. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  14. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  15. 2016Understanding Neural Networks through Representation Erasurecited by 223
  16. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 216
  17. 2019Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representationcited by 210
  18. 2019SPoC: Search-based Pseudocode to Codecited by 176
  19. 2021Retrieval Augmented Code Generation and Summarizationcited by 148
  20. 2019Fine-Grained Analysis of Propaganda in News Articlescited by 130
  21. 2020Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERTcited by 109
  22. 2019Attention in Natural Language Processingcited by 100
  23. 2020Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learningcited by 99
  24. 2022CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generationcited by 87
  25. 2020Contrastive Code Representation Learningcited by 76
  26. 2019Attention Interpretability Across NLP Taskscited by 71
  27. 2021Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Taskscited by 70
  28. 2022Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?cited by 63
  29. 2019Adversarial Examples for Models of Codecited by 61
  30. 2022NatGen: Generative pre-training by "Naturalizing" source codecited by 53
  31. 2014Extraction of Salient Sentences from Labelled Documentscited by 50
  32. 2022What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source Codecited by 50
  33. 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codexcited by 41
  34. 2023SkCoder: A Sketch-based Approach for Automatic Code Generationcited by 40
  35. 2020Adversarial Robustness for Codecited by 36
  36. 2021What do pre-trained code models know about code?cited by 36
  37. 2016Joint Attention in Autonomous Driving (JAAD)cited by 35
  38. 2021Shared Interest: Measuring Human-AI Alignment to Identify Recurring Patterns in Model Behaviorcited by 29

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 4.

13 papers.

Sort:

By area:

  1. 2025Correctness Assessment of Code Generated by Large Language Models Using Internal Representationscited by 17
  2. 2024Selective Prompt Anchoring for Code Generationcited by 16
  3. 2024LLMs: A Game-Changer for Software Engineers?cited by 11
  4. 2024A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?cited by 9
  5. 2025Risk Assessment Framework for Code LLMs via Leveraging Internal Statescited by 8
  6. 2024A Survey on Large Language Models from Concept to Implementationcited by 4
  7. 2025Enhancing Code Generation via Bidirectional Comment-Level Mutual Groundingcited by 3
  8. 2026Agent Skill Framework: Perspectives on the Potential of Small to Medium Language Models in Industrial Environmentscited by 3
  9. 2026SynthFix: Adaptive Neuro-Symbolic Code Vulnerability Repaircited by 1
  10. 2025Enhancing Code LLM Training with Programmer Attentioncited by 0
  11. 2025Testing chatbots on the creation of encoders for audio conditioned image generationcited by 0
  12. 2026Code-MUE: Measuring Code LLMs' Uncertainty through Execution-based Semantic Interaction Graphscited by 0
1 more
  1. 2026How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repaircited by 0