A Comparative Study of Code Generation using ChatGPT 3.5 across 10 Programming Languages
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) are advanced Artificial Intelligence (AI) systems that have undergone extensive training using large datasets in order to understand and produce language that closely resembles that of humans. These models have reached a level of proficiency where they are capable of successfully completing university exams across several disciplines and generating functional code to handle novel problems. This research investigates the coding proficiency of ChatGPT 3.5, a LLM released by OpenAI in November 2022, which has gained significant recognition for its impressive text generating and code creation capabilities. The skill of the model in creating code snippets is evaluated across 10 various programming languages and 4 different software domains. Based on the findings derived from this research, major unexpected behaviors and limitations of the model have been identified. This study aims to identify potential areas for development and examine the ramifications of automated code generation on the evolution of programming languages and on the tech industry.
Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Aug 2023.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2023: 3; 2024: 5; 2025: 3.
11 papers.
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
- 2023Autonomous Agents in Software Development: A Vision Papercited by 12
- 2024AutoCoder: Enhancing Code Large Language Model with textscAIEV-Instructcited by 8
- 2023Automating the Correctness Assessment of AI-generated Code for Security Contextscited by 6
- 2025Large Language Models for In-File Vulnerability Localization Can Be "Lost in the End"cited by 5
- 2024RogueGPT: dis-ethical tuning transforms ChatGPT4 into a Rogue AI in 158 Wordscited by 4
- 2025The Impact of LLM-Assistants on Software Developer Productivity: A Systematic Review and Mapping Studycited by 4
- 2024Using Large Language Models to Generate Clinical Trial Tables and Figurescited by 3
- 2023Leveraging Large Language Models to Build and Execute Computational Workflowscited by 1
- 2024Can OpenSource beat ChatGPT? -- A Comparative Study of Large Language Models for Text-to-Code Generationcited by 0
- 2025A Systematic Evaluation of Parameter-Efficient Fine-Tuning Methods for the Security of Code LLMscited by 0
- 2025A Systematic Evaluation of Parameter-Efficient Fine-Tuning Methods for the Security of Code LLMs16 Sep 2025
- 2025The Impact of LLM-Assistants on Software Developer Productivity: A Systematic Review and Mapping Study3 Jul 2025
- 2025Large Language Models for In-File Vulnerability Localization Can Be "Lost in the End"9 Feb 2025
- 2024Using Large Language Models to Generate Clinical Trial Tables and Figures18 Sep 2024
- 2024Can OpenSource beat ChatGPT? -- A Comparative Study of Large Language Models for Text-to-Code Generation6 Sep 2024
- 2024RogueGPT: dis-ethical tuning transforms ChatGPT4 into a Rogue AI in 158 Words11 Jun 2024
- 2024AutoCoder: Enhancing Code Large Language Model with textscAIEV-Instruct23 May 2024
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Review6 May 2024
- 2023Leveraging Large Language Models to Build and Execute Computational Workflows12 Dec 2023
- 2023Autonomous Agents in Software Development: A Vision Paper30 Nov 2023
- 2023Automating the Correctness Assessment of AI-generated Code for Security Contexts28 Oct 2023