# Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?

arXiv: https://arxiv.org/abs/2306.01220 · HTML: https://mlsanity.com/paper/2306.01220/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 67 of its 105 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Builds on (most cited first, 30 of 67)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [Aligning AI With Shared Human Values](https://mlsanity.com/paper/2008.02275/) (2020)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](https://mlsanity.com/paper/2203.14680/) (2022)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [Unified Pre-training for Program Understanding and Generation](https://mlsanity.com/paper/2103.06333/) (2021)
- [A Systematic Evaluation of Large Language Models of Code](https://mlsanity.com/paper/2202.13169/) (2022)
- [Understanding Neural Networks through Representation Erasure](https://mlsanity.com/paper/1612.08220/) (2016)
- [Grounded Copilot: How Programmers Interact with Code-Generating Models](https://mlsanity.com/paper/2206.15000/) (2022)
- [Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation](https://mlsanity.com/paper/1905.08205/) (2019)
- [SPoC: Search-based Pseudocode to Code](https://mlsanity.com/paper/1906.04908/) (2019)

## Built on by (most cited first, 14 of 14)

- [Correctness Assessment of Code Generated by Large Language Models Using Internal Representations](https://mlsanity.com/paper/2501.12934/) (2025)
- [Selective Prompt Anchoring for Code Generation](https://mlsanity.com/paper/2408.09121/) (2024)
- [LLMs: A Game-Changer for Software Engineers?](https://mlsanity.com/paper/2411.00932/) (2024)
- [A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why?](https://mlsanity.com/paper/2411.01414/) (2024)
- [Risk Assessment Framework for Code LLMs via Leveraging Internal States](https://mlsanity.com/paper/2504.14640/) (2025)
- [A Survey on Large Language Models from Concept to Implementation](https://mlsanity.com/paper/2403.18969/) (2024)
- [Enhancing Code Generation via Bidirectional Comment-Level Mutual Grounding](https://mlsanity.com/paper/2505.07768/) (2025)
- [Agent Skill Framework: Perspectives on the Potential of Small to Medium Language Models in Industrial Environments](https://mlsanity.com/paper/2602.16653/) (2026)
- [SynthFix: Adaptive Neuro-Symbolic Code Vulnerability Repair](https://mlsanity.com/paper/2604.17184/) (2026)
- [Enhancing Code LLM Training with Programmer Attention](https://mlsanity.com/paper/2503.14936/) (2025)
- [Testing chatbots on the creation of encoders for audio conditioned image generation](https://mlsanity.com/paper/2509.09717/) (2025)
- [Code-MUE: Measuring Code LLMs' Uncertainty through Execution-based Semantic Interaction Graphs](https://mlsanity.com/paper/2607.12273/) (2026)
- [How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair](https://mlsanity.com/paper/2607.25873/) (2026)
- [Keep Evaluation Fair: Detecting Data Leakage in Code Generation Benchmarks via Membership Inference Attacks](https://mlsanity.com/paper/2609.09865/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have recently been widely used for code generation. Due to the complexity and opacity of LLMs, little is known about how these models generate code. We made the first attempt to bridge this knowledge gap by investigating whether LLMs attend to the same parts of a task description as human programmers during code generation. An analysis of six LLMs, including GPT-4, on two popular code generation benchmarks revealed a consistent misalignment between LLMs' and programmers' attention. We manually analyzed 211 incorrect code snippets and found five attention patterns that can be used to explain many code generation errors. Finally, a user study showed that model attention computed by a perturbation-based method is often favored by human programmers. Our findings highlight the need for human-aligned LLMs for better interpretability and programmer trust.