mlsanityData to 7 Oct 2026

Evaluating the Application of Large Language Models to Generate Feedback in Programming Education

arXiv 2403.09744 ↗First posted 13 Mar 2024cs.CL cs.AI cs.CYcited by 10

Not on the map. Why?

Abstract (arXiv)

This study investigates the application of large language models, specifically GPT-4, to enhance programming education. The research outlines the design of a web application that uses GPT-4 to provide feedback on programming tasks, without giving away the solution. A web application for working on programming tasks was developed for the study and evaluated with 51 students over the course of one semester. The results show that most of the feedback generated by GPT-4 effectively addressed code errors. However, challenges with incorrect suggestions and hallucinated issues indicate the need for further improvements.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 15 of its 23 references that are in the map. First posted on arXiv 13 Mar 2024.

Builds on

15 of 23 references.

References · 15

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  3. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  4. 2023The Robots are Here: Navigating the Generative AI Revolution in Computing Educationcited by 85
  5. 2024CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needscited by 81
  6. 2023CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classescited by 71
  7. 2023Computing Education in the Era of Generative AIcited by 50
  8. 2023Exploring the Responses of Large Language Models to Beginner Programmers' Help Requestscited by 44
  9. 2023Generative AI for Programming Education: Benchmarking ChatGPT, GPT-4, and Human Tutorscited by 38
  10. 2023Large Language Models (GPT) for automating feedback on programming assignmentscited by 32
  11. 2023Exploring the Potential of Large Language Models to Generate Formative Programming Feedbackcited by 28
  12. 2023Next-Step Hint Generation for Introductory Programming Using Large Language Modelscited by 20
3 more
  1. 2024Feedback-Generation for Programming Exercises With GPT-4cited by 14
  2. 2023Large Language Models in Introductory Programming Education: ChatGPT's Performance and Implications for Assessmentscited by 10
  3. 2023AI-enhanced Auto-correction of Programming Exercises: How Effective is GPT-3.5?cited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 4; 2026: 2.

10 papers.

  1. 2025Rubric Is All You Need: Enhancing LLM-based Code Evaluation With Question-Specific Rubricscited by 29
  2. 2024Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Toolscited by 13
  3. 2024Large Language Models in Computer Science Education: A Systematic Literature Reviewcited by 12
  4. 2024Grading Massive Open Online Courses Using Large Language Modelscited by 7
  5. 2024Leveraging Lecture Content for Improved Feedback: Explorations with GPT-4 and Retrieval Augmented Generationcited by 6
  6. 2025Assessing Large Language Models for Automated Feedback Generation in Learning Programming Problem Solvingcited by 2
  7. 2025Students' Feedback Requests and Interactions with the SCRIPT Chatbot: Do They Get What They Ask For?cited by 1
  8. 2025GenAI Voice Mode in Programming Educationcited by 0
  9. 2026An Explainable AI Assistant for Introductory Programming Education: Improving Feedback Reliability with Instructor-AI Collaborationcited by 0
  10. 2026Automated grading of Linux/bash examinations using large language models: a four-level cognitive taxonomy approachcited by 0