mlsanityData to 7 Oct 2026

Large Language Models (LLMs) for Source Code Analysis: applications, models and datasets

arXiv 2503.17502 ↗First posted 21 Mar 2025cs.SE cs.AI cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) and transformer-based architectures are increasingly utilized for source code analysis. As software systems grow in complexity, integrating LLMs into code analysis workflows becomes essential for enhancing efficiency, accuracy, and automation. This paper explores the role of LLMs for different code analysis tasks, focusing on three key aspects: 1) what they can analyze and their applications, 2) what models are used and 3) what datasets are used, and the challenges they face. Regarding the goal of this research, we investigate scholarly articles that explore the use of LLMs for source code analysis to uncover research developments, current trends, and the intellectual structure of this emerging field. Additionally, we summarize limitations and highlight essential tools, datasets, and key challenges, which could be valuable for future work.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 91 of its 163 references that are in the map. First posted on arXiv 21 Mar 2025.

Builds on

91 of 163 references.

References · 91

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  3. 2020Longformer: The Long-Document Transformercited by 3,362
  4. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  5. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  6. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  7. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  8. 2021Measuring Coding Challenge Competence With APPScited by 879
  9. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  10. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  11. 2024A Survey on Large Language Models for Code Generationcited by 646
  12. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
38 more
  1. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  2. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  3. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  4. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  5. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  6. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  7. 2020IntelliCode Compose: Code Generation Using Transformercited by 225
  8. 2023Large Language Models for Software Engineering: Survey and Open Problemscited by 178
  9. 2023LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarkscited by 94
  10. 2025ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generationcited by 88
  11. 2022CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generationcited by 87
  12. 2021CURE: Code-Aware Neural Machine Translation for Automatic Program Repaircited by 85
  13. 2024Planning In Natural Language Improves LLM Search For Code Generationcited by 79
  14. 2023Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)cited by 72
  15. 2023Large Language Models are Few-Shot Summarizers: Multi-Intent Comment Generation via In-Context Learningcited by 71
  16. 2023Towards an Understanding of Large Language Models in Software Engineering Taskscited by 67
  17. 2023A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trendscited by 66
  18. 2024A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languagescited by 65
  19. 2023LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluationscited by 58
  20. 2022NatGen: Generative pre-training by "Naturalizing" source codecited by 53
  21. 2024LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluationcited by 46
  22. 2022jTrans: Jump-Aware Transformer for Binary Code Similaritycited by 44
  23. 2024Evolving Code with A Large Language Modelcited by 42
  24. 2021TreeBERT: A Tree-Based Pre-Trained Model for Programming Languagecited by 40
  25. 2023Explainable Automated Debugging via Large Language Model-driven Scientific Debuggingcited by 40
  26. 2024LLM4Decompile: Decompiling Binary Code with Large Language Modelscited by 38
  27. 2023SALLM: Security Assessment of Generated Codecited by 35
  28. 2024RepoQA: Evaluating Long Context Code Understandingcited by 35
  29. 2024Test-Driven Development for Code Generationcited by 32
  30. 2023DeceptPrompt: Exploiting LLM-driven Code Generation via Adversarial Natural Language Instructionscited by 31
  31. 2023Distilled GPT for Source Code Summarizationcited by 30
  32. 2024Source Code Summarization in the Era of Large Language Modelscited by 30
  33. 2024CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?cited by 29
  34. 2023CoderEval: A Benchmark of Pragmatic Code Generation with Generative Pre-trained Modelscited by 28
  35. 2023Large Language Models in Fault Localisationcited by 28
  36. 2024Toward Secure Code Generation: Bridging Correctness and Security via Task-Adaptive Vulnerability Modeling and Execution-Based Benchmarkingcited by 26
  37. 2023Refining Decompiled C Code with Large Language Modelscited by 24
  38. 2023A Comparative Analysis of Large Language Models for Code Documentation Generationcited by 24

50 most cited of 91 All 91 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 6.

15 papers.

Sort:

By area:

  1. 2026Repository Intelligence Graph: Deterministic Architectural Map for LLM Code Assistantscited by 4
  2. 2025Trust Me, I Know This Function: Hijacking LLM Static Analysis using Biascited by 3
  3. 2025A Blueprint for AI-Driven Software Quality: Integrating LLMs with Established Standardscited by 1
  4. 2025OmniLLP: Enhancing LLM-based Log Level Prediction with Context-Aware Retrievalcited by 1
  5. 2025NLD-LLM: A systematic framework for evaluating small language transformer models on natural language descriptioncited by 1
  6. 2025Hound: Relation-First Knowledge Graphs for Complex-System Reasoning in Security Auditscited by 1
  7. 2025Divided We Fall: Defending Against Adversarial Attacks via Soft-Gated Fractional Mixture-of-Experts with Randomized Adversarial Trainingcited by 1
  8. 2026Automated Malware Family Classification using Weighted Hierarchical Ensembles of Large Language Modelscited by 1
  9. 2026ORBIT: Guided Agentic Orchestration for Autonomous C-to-Rust Transpilationcited by 1
  10. 2025XGen-Q: An Explainable Domain-Adaptive LLM Framework with Retrieval-Augmented Generation for Software Securitycited by 0
  11. 2025FlexiDataGen: An Adaptive LLM Framework for Dynamic Semantic Dataset Generation in Sensitive Domainscited by 0
  12. 2025Asm2SrcEval: Evaluating Large Language Models for Assembly-to-Source Code Translationcited by 0
3 more
  1. 2026LLM4CodeRE: Generative AI for Code Decompilation Analysis and Reverse Engineeringcited by 0
  2. 2026LLM-Based Code Documentation Generation and Multi-Judge Evaluationcited by 0
  3. 2026Beyond the Autoregressive Horizon: A Comprehensive Survey of Diffusion Models, World Modelling, and State Space Models for Codecited by 0