Commenting with Copilot: A Taxonomy and Multi-Year Analysis of Student Code-Generation Specifications
Not on the map. Why?
Abstract (arXiv)
As AI code tools become integrated into programming environments, students increasingly describe intended behavior in natural language and rely on these tools to generate code, shifting emphasis from code writing to specification. Yet little is known about the comments students write as specifications in AI-assisted programming tasks. We analyze a four-year dataset of undergraduate programming submissions and reflections from tasks in which students wrote comments to guide code generation and refined solutions using test-case feedback. We introduce a taxonomy spanning three dimensions: comment type, code expression level, and code construct. Using automated classification, we examine how these dimensions vary across attempts and how students describe the process in their reflections. Our findings show that students mostly wrote natural-language What comments, shifted toward How comments for more procedural constructs, and focused more on verifying generated code than on repeatedly rewriting comments.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 41 references that are in the map. First posted on arXiv 12 Jul 2026.
Builds on
12 of 41 references.
References · 12
- 2023LLM-in-the-loop: Leveraging Large Language Model for Thematic Analysiscited by 70
- 2023LLM-Assisted Content Analysis: Using Large Language Models to Support Deductive Codingcited by 64
- 2023How Novices Use LLM-Based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning Environmentcited by 56
- 2023Prompt Problems: A New Programming Exercise for the Generative AI Eracited by 31
- 2024How Beginning Programmers and Code LLMs (Mis)read Each Othercited by 31
- 2024The Future of AI-Driven Software Engineeringcited by 23
- 2024Explaining Code with a Purpose: An Integrated Approach for Developing Code Comprehension and Prompting Skillscited by 11
- 2025Evolution of Programmers' Trust in Generative AI Programming Assistantscited by 3
- 2023Code Generation Based Grading: Evaluating an Auto-grading Mechanism for "Explain-in-Plain-English" Questionscited by 2
- 2024Probeable Problems for Beginner-level Programming-with-AI Contestscited by 2
- 2024Breaking the Programming Language Barrier: Multilingual Prompting to Empower Non-Native English Learnerscited by 2
- 2024Explain in Plain Language Questions with Indic Languages: Drawbacks, Affordances, and Opportunitiescited by 1
Built on by
Not cited yet.