mlsanityData to 7 Oct 2026

What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source Code

arXiv 2202.06840 ↗First posted 14 Feb 2022cs.SE cs.AIcited by 50

Not on the map. Why?

Abstract (arXiv)

Recently, many pre-trained language models for source code have been proposed to model the context of code and serve as a basis for downstream code intelligence tasks such as code completion, code search, and code summarization. These models leverage masked pre-training and Transformer and have achieved promising results. However, currently there is still little progress regarding interpretability of existing pre-trained code models. It is not clear why these models work and what feature correlations they can capture. In this paper, we conduct a thorough structural analysis aiming to provide an interpretation of pre-trained language models for source code (e.g., CodeBERT, and GraphCodeBERT) from three distinctive perspectives: (1) attention analysis, (2) probing on the word embedding, and (3) syntax tree induction. Through comprehensive analysis, this paper reveals several insightful findings that may inspire future studies: (1) Attention aligns strongly with the syntax structure of code. (2) Pre-training language models of code can preserve the syntax structure of code in the intermediate representations of each Transformer layer. (3) The pre-trained models of code have the ability of inducing syntax trees of code. Theses findings suggest that it may be helpful to incorporate the syntax structure of code into the process of pre-training for better code representations.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 14 Feb 2022.

Builds on

33 of 45 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  8. 2018Deep contextualized word representationscited by 2,285
  9. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  10. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  11. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  12. 2020A Primer in BERTology: What we know about how BERT workscited by 807
21 more
  1. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  2. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
  3. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  4. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  5. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  6. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  7. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  8. 2019ERNIE: Enhanced Representation through Knowledge Integrationcited by 301
  9. 2019Revealing the Dark Secrets of BERTcited by 300
  10. 2018code2vec: Learning Distributed Representations of Codecited by 249
  11. 2019Analyzing the Structure of Attention in a Transformer Language Modelcited by 241
  12. 2018code2seq: Generating Sequences from Structured Representations of Codecited by 177
  13. 2018Improving Automatic Source Code Summarization via Deep Reinforcement Learningcited by 99
  14. 2019Do Attention Heads in BERT Track Syntactic Dependencies?cited by 87
  15. 2020BERTology Meets Biology: Interpreting Attention in Protein Language Modelscited by 85
  16. 2020Are Pre-trained Language Models Aware of Phrases? Simple but Strong Baselines for Grammar Inductioncited by 47
  17. 2019Multi-Modal Attention Network Learning for Semantic Source Code Retrievalcited by 40
  18. 2019exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Modelscited by 33
  19. 2018Straight to the Tree: Constituency Parsing with Neural Syntactic Distancecited by 32
  20. 2019What do you mean, BERT? Assessing BERT as a Distributional Semantics Modelcited by 14
  21. 2020Empirical Study of Transformers for Source Codecited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 16; 2024: 16; 2025: 7; 2026: 6.

50 papers.

Sort:

By area:

  1. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  2. 2024Astraios: Parameter-Efficient Instruction Tuning Code Large Language Modelscited by 23
  3. 2023Deep Learning for Code Intelligence: Survey, Benchmark and Toolkitcited by 22
  4. 2023Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyondcited by 20
  5. 2023Exploring Code Analysis: Zero-Shot Insights on Syntax and Semantics with LLMscited by 19
  6. 2022AST-Probe: Recovering abstract syntax trees from hidden representations of pre-trained language modelscited by 16
  7. 2024Selective Prompt Anchoring for Code Generationcited by 16
  8. 2022CODE-MVP: Learning to Represent Source Code from Multiple Views with Contrastive Pre-Trainingcited by 15
  9. 2024From Imitation to Introspection: Probing Self-Consciousness in Language Modelscited by 14
  10. 2023Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code?cited by 13
  11. 2024Graph Neural Networks for Vulnerability Detection: A Counterfactual Explanationcited by 12
  12. 2024Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanationscited by 10
38 more
  1. 2024RAMBO: Enhancing RAG-based Repository-Level Method Body Completioncited by 10
  2. 2022Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacitiescited by 9
  3. 2023Evaluating and Explaining Large Language Models for Code Using Syntactic Structurescited by 9
  4. 2023Pitfalls in Language Models for Code Intelligence: A Taxonomy and Surveycited by 9
  5. 2023Turbulence: Systematically and Automatically Testing Instruction-Tuned Large Language Models for Codecited by 9
  6. 2024An Empirical Study on Capability of Large Language Models in Understanding Code Semanticscited by 7
  7. 2022CAT-probing: A Metric-based Approach to Interpret How Pre-trained Models for Programming Language Attend Code Structurecited by 6
  8. 2022Follow-up Attention: An Empirical Study of Developer and Neural Model Code Explorationcited by 5
  9. 2024Looking into Black Box Code Language Modelscited by 5
  10. 2023Towards Trustworthy AI Software Development Assistancecited by 4
  11. 2024A Critical Study of What Code-LLMs (Do Not) Learncited by 4
  12. 2024Showing LLM-Generated Code Selectively Based on Confidence of LLMscited by 4
  13. 2024Do Current Language Models Support Code Intelligence for R Programming Language?cited by 4
  14. 2025How Programming Concepts and Neurons Are Shared in Code Language Modelscited by 4
  15. 2023Towards Understanding What Code Language Models Learnedcited by 3
  16. 2023Pop Quiz! Do Pre-trained Code Models Possess Knowledge of Correct API Names?cited by 3
  17. 2025Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearningcited by 3
  18. 2025Neuron-Guided Interpretation of Code LLMs: Where, Why, and How?cited by 3
  19. 2023APIContext2Com: Code Comment Generation by Incorporating Pre-Defined API Documentationcited by 2
  20. 2023INSPECT: Intrinsic and Systematic Probing Evaluation for Code Transformerscited by 2
  21. 2024A Systematic Literature Review on Explainability for Machine/Deep Learning-based Software Engineering Researchcited by 2
  22. 2026DeepGuard: Secure Code Generation via Multi-Layer Semantic Aggregationcited by 2
  23. 2023Redundancy and Concept Analysis for Code-trained Language Modelscited by 1
  24. 2023Naturalness of Attention: Revisiting Attention in Code Language Modelscited by 1
  25. 2024Beyond Self-learned Attention: Mitigating Attention Bias in Transformer-based Models Using Attention Guidancecited by 1
  26. 2024DeepCodeProbe: Towards Understanding What Models Trained on Code Learncited by 1
  27. 2025Analyzing Latent Concepts in Code Language Modelscited by 1
  28. 2026Towards Understanding What State Space Models Learn About Codecited by 1
  29. 2026CSP-Atlas: Concept-Specific Neural Circuits in a Sparse Python Transformercited by 1
  30. 2023Towards Automatic Support of Software Model Evolution with Large Language~Modelscited by 0
  31. 2024In-Context Code-Text Learning for Bimodal Software Engineeringcited by 0
  32. 2024Embedding with Large Language Models for Classification of HIPAA Safeguard Compliance Rulescited by 0
  33. 2025How to Select Pre-Trained Code Models for Reuse? A Learning Perspectivecited by 0
  34. 2025Mapping the Trust Terrain: LLMs in Software Engineering -- Insights and Perspectivescited by 0
  35. 2025LEANCODE: Understanding Models Better for Code Simplification of Pre-trained Large Language Modelscited by 0
  36. 2026Co-Located Tests, Better AI Code: How Test Syntax Structure Affects Foundation Model Code Generationcited by 0
  37. 2026Latent Programming Horizons in Coding Agentscited by 0
  38. 2026What, Where, and How: Disentangling the Roles of Task, Language, and Model in Code Model Representationscited by 0