Evaluating the Quality of Code Comments Generated by Large Language Models for Novice Programmers
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) show promise in generating code comments for novice programmers, but their educational effectiveness remains under-evaluated. This study assesses the instructional quality of code comments produced by GPT-4, GPT-3.5-Turbo, and Llama2, compared to expert-developed comments, focusing on their suitability for novices. Analyzing a dataset of ``easy'' level Java solutions from LeetCode, we find that GPT-4 exhibits comparable quality to expert comments in aspects critical for beginners, such as clarity, beginner-friendliness, concept elucidation, and step-by-step guidance. GPT-4 outperforms Llama2 in discussing complexity (chi-square = 11.40, p = 0.001) and is perceived as significantly more supportive for beginners than GPT-3.5 and Llama2 with Mann-Whitney U-statistics = 300.5 and 322.5, p = 0.0017 and 0.0003). This study highlights the potential of LLMs for generating code comments tailored to novice programmers.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 14 references that are in the map. First posted on arXiv 22 Sep 2024.
Builds on
8 of 14 references.
References · 8
- 2022A Systematic Evaluation of Large Language Models of Codecited by 363
- 2022Language Models of Code are Few-Shot Commonsense Learnerscited by 160
- 2022Using Large Language Models to Enhance Programming Error Messagescited by 67
- 2023Comparing Code Explanations Created by Students and Large Language Modelscited by 64
- 2023Explaining Competitive-Level Programming Solutions using LLMscited by 13
- 2022Automatically Generating CS Learning Materials with Large Language Modelscited by 12
- 2022CodeExp: Explanatory Code Document Generationcited by 4
- 2023Exploring the Potential of Large Language Models in Generating Code-Tracing Questions for Introductory Programming Coursescited by 3
Built on by
Not cited yet.