# Automated Grading and Feedback Tools for Programming Education: A Systematic Review

arXiv: https://arxiv.org/abs/2306.11722 · HTML: https://mlsanity.com/paper/2306.11722/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2023.

## Built on by (most cited first, 17 of 17)

- [Rubric Is All You Need: Enhancing LLM-based Code Evaluation With Question-Specific Rubrics](https://mlsanity.com/paper/2503.23989/) (2025)
- [A large language model-assisted education tool to provide feedback on open-ended responses](https://mlsanity.com/paper/2308.02439/) (2023)
- [Leveraging Large Language Model as Simulated Patients for Clinical Education](https://mlsanity.com/paper/2404.13066/) (2024)
- [LLM-based Automated Grading with Human-in-the-Loop](https://mlsanity.com/paper/2504.05239/) (2025)
- [Evaluating Language Models for Generating and Judging Programming Feedback](https://mlsanity.com/paper/2407.04873/) (2024)
- [Survey of Natural Language Processing for Education: Taxonomy, Systematic Review, and Future Trends](https://mlsanity.com/paper/2401.07518/) (2024)
- [CodEv: An Automated Grading Framework Leveraging Large Language Models for Consistent and Constructive Feedback](https://mlsanity.com/paper/2501.10421/) (2025)
- [ARISE: Agentic Rubric-Guided Iterative Survey Engine for Automated Scholarly Paper Generation](https://mlsanity.com/paper/2511.17689/) (2025)
- [Teaching LLMs for Step-Level Automatic Math Correction via Reinforcement Learning](https://mlsanity.com/paper/2503.18432/) (2025)
- ["Bespoke Bots": Diverse Instructor Needs for Customizing Generative AI Classroom Chatbots](https://mlsanity.com/paper/2603.00057/) (2026)
- [Comparing Developer and LLM Biases in Code Evaluation](https://mlsanity.com/paper/2603.24586/) (2026)
- [LLM-Driven Personalized Answer Generation and Evaluation](https://mlsanity.com/paper/2506.10829/) (2025)
- [Autograder+: A Multi-Faceted AI Framework for Rich Pedagogical Feedback in Programming Education](https://mlsanity.com/paper/2510.26402/) (2025)
- [Trustworthy Intelligent Education: A Systematic Perspective on Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2601.21837/) (2026)
- [An Explainable AI Assistant for Introductory Programming Education: Improving Feedback Reliability with Instructor-AI Collaboration](https://mlsanity.com/paper/2606.12425/) (2026)
- [CAPRA: Scaling Feedback on Software Architecture Deliverables with a Multi-Agent LLM System](https://mlsanity.com/paper/2606.18976/) (2026)
- [Where LLM Graders Succeed and Break: Evidence from Two Computer-Science Exams](https://mlsanity.com/paper/2609.29333/) (2026)

## Abstract (arXiv)

We conducted a systematic literature review on automated grading and feedback tools for programming education. We analysed 121 research papers from 2017 to 2021 inclusive and categorised them based on skills assessed, approach, language paradigm, degree of automation and evaluation techniques. Most papers assess the correctness of assignments in object-oriented languages. Typically, these tools use a dynamic technique, primarily unit testing, to provide grades and feedback to the students or static analysis techniques to compare a submission with a reference solution or with a set of correct student submissions. However, these techniques' feedback is often limited to whether the unit tests have passed or failed, the expected and actual output, or how they differ from the reference solution. Furthermore, few tools assess the maintainability, readability or documentation of the source code, with most using static analysis techniques, such as code quality metrics, in conjunction with grading correctness. Additionally, we found that most tools offered fully automated assessment to allow for near-instantaneous feedback and multiple resubmissions, which can increase student satisfaction and provide them with more opportunities to succeed. In terms of techniques used to evaluate the tools' performance, most papers primarily use student surveys or compare the automatic assessment tools to grades or feedback provided by human graders. However, because the evaluation dataset is frequently unavailable, it is more difficult to reproduce results and compare tools to a collection of common assignments.