# Students' Perceptions and Preferences of Generative Artificial Intelligence Feedback for Programming

arXiv: https://arxiv.org/abs/2312.11567 · HTML: https://mlsanity.com/paper/2312.11567/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 36 references that are in the map.
- First posted on arXiv 17 Dec 2023.

## Builds on (most cited first, 8 of 8)

- [Learning gain differences between ChatGPT and human tutor generated algebra hints](https://mlsanity.com/paper/2302.06871/) (2023)
- [Large Language Models (GPT) for automating feedback on programming assignments](https://mlsanity.com/paper/2307.00150/) (2023)
- [Automated Clustering and Program Repair for Introductory Programming Assignments](https://mlsanity.com/paper/1603.03165/) (2016)
- [Learning Program Embeddings to Propagate Feedback on Student Code](https://mlsanity.com/paper/1505.05969/) (2015)
- [Code-DKT: A Code-based Knowledge Tracing Model for Programming Tasks](https://mlsanity.com/paper/2206.03545/) (2022)
- [Toward Semi-Automatic Misconception Discovery Using Code Embeddings](https://mlsanity.com/paper/2103.04448/) (2021)
- [A Comparison of Inquiry-Based Conceptual Feedback vs. Traditional Detailed Feedback Mechanisms in Software Testing Education: An Empirical Investigation](https://mlsanity.com/paper/2011.13004/) (2020)
- [Investigating the Essential of Meaningful Automated Formative Feedback for Programming Assignments](https://mlsanity.com/paper/1906.08937/) (2019)

## Built on by (most cited first, 6 of 6)

- [Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Tools](https://mlsanity.com/paper/2412.14732/) (2024)
- [Grading Massive Open Online Courses Using Large Language Models](https://mlsanity.com/paper/2406.11102/) (2024)
- [Scaffolding Metacognition in Programming Education: Understanding Student-AI Interactions and Design Implications](https://mlsanity.com/paper/2511.04144/) (2025)
- [Evaluating Trust in AI, Human, and Co-produced Feedback Among Undergraduate Students](https://mlsanity.com/paper/2504.10961/) (2025)
- [AICoFe: Implementation and Deployment of an AI-Based Collaborative Feedback System for Higher Education](https://mlsanity.com/paper/2605.04740/) (2026)
- [AI-Assisted Help-Seeking Trajectories in Programming Education from an SRL-Informed Perspective](https://mlsanity.com/paper/2606.22809/) (2026)

## Abstract (arXiv)

The rapid evolution of artificial intelligence (AI), specifically large language models (LLMs), has opened opportunities for various educational applications. This paper explored the feasibility of utilizing ChatGPT, one of the most popular LLMs, for automating feedback for Java programming assignments in an introductory computer science (CS1) class. Specifically, this study focused on three questions: 1) To what extent do students view LLM-generated feedback as formative? 2) How do students see the comparative affordances of feedback prompts that include their code, vs. those that exclude it? 3) What enhancements do students suggest for improving AI-generated feedback? To address these questions, we generated automated feedback using the ChatGPT API for four lab assignments in the CS1 class. The survey results revealed that students perceived the feedback as aligning well with formative feedback guidelines established by Shute. Additionally, students showed a clear preference for feedback generated by including the students' code as part of the LLM prompt, and our thematic study indicated that the preference was mainly attributed to the specificity, clarity, and corrective nature of the feedback. Moreover, this study found that students generally expected specific and corrective feedback with sufficient code examples, but had diverged opinions on the tone of the feedback. This study demonstrated that ChatGPT could generate Java programming assignment feedback that students perceived as formative. It also offered insights into the specific improvements that would make the ChatGPT-generated feedback useful for students.