mlsanityData to 14 Oct 2026

Analysis of ChatGPT on Source Code

arXiv 2306.00597 ↗First posted 1 Jun 2023cs.SE cs.AI cs.PLcited by 12

Not on the map. Why?

Abstract (arXiv)

This paper explores the use of Large Language Models (LLMs) and in particular ChatGPT in programming, source code analysis, and code generation. LLMs and ChatGPT are built using machine learning and artificial intelligence techniques, and they offer several benefits to developers and programmers. While these models can save time and provide highly accurate results, they are not yet advanced enough to replace human programmers entirely. The paper investigates the potential applications of LLMs and ChatGPT in various areas, such as code creation, code documentation, bug detection, refactoring, and more. The paper also suggests that the usage of LLMs and ChatGPT is expected to increase in the future as they offer unparalleled benefits to the programming community.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 6 of its 17 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

6 of 17 references.

References · 6

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2023A Categorical Archive of ChatGPT Failurescited by 168
  3. 2023An Analysis of the Automatic Bug Fixing Performance of ChatGPTcited by 110
  4. 2023ChatGPT is not all you need. A State of the Art Review of large Generative AI modelscited by 86
  5. 2023Memory Augmented Large Language Models are Computationally Universalcited by 47
  6. 2023A Glimpse in ChatGPT Capabilities and its impact for AI researchcited by 6

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 5; 2025: 4.

12 papers.

  1. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 68
  2. 2023Testing the Depth of ChatGPT's Comprehension via Cross-Modal Tasks Based on ASCII-Art: GPT3.5's Abilities in Regard to Recognizing and Generating ASCII-Art Are Not Totally Lackingcited by 7
  3. 2023Coding by Design: GPT-4 empowers Agile Model Driven Developmentcited by 6
  4. 2024Can LLMs be Fooled? Investigating Vulnerabilities in LLMscited by 5
  5. 2025Benchmarking LLM for Code Smells Detection: OpenAI GPT-4.0 vs DeepSeek-V3cited by 5
  6. 2023LLM Interactive Optimization of Open Source Python Libraries -- Case Studies and Generalizationcited by 3
  7. 2024CodeCloak: A Method for Evaluating and Mitigating Code Leakage by LLM Code Assistantscited by 3
  8. 2025SOK: Exploring Hallucinations and Security Risks in AI-Assisted Software Development with Insights for LLM Deploymentcited by 2
  9. 2025Human-in-the-Loop: Quantitative Evaluation of 3D Models Generation by Large Language Modelscited by 2
  10. 2024LLM as a code generator in Agile Model Driven Developmentcited by 1
  11. 2025Can We Make Code Green? Understanding Trade-Offs in LLMs vs. Human Code Optimizationscited by 1
  12. 2024HyperGraphOS: A Meta Operating System for Science and Engineeringcited by 0