# Autograder+: A Multi-Faceted AI Framework for Rich Pedagogical Feedback in Programming Education

arXiv: https://arxiv.org/abs/2510.26402 · HTML: https://mlsanity.com/paper/2510.26402/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 55 references that are in the map.
- First posted on arXiv 30 Oct 2025.

## Builds on (most cited first, 22 of 22)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [CodeT5+: Open Code Large Language Models for Code Understanding and Generation](https://mlsanity.com/paper/2305.07922/) (2023)
- [OpenCodeReasoning: Advancing Data Distillation for Competitive Coding](https://mlsanity.com/paper/2504.01943/) (2025)
- [Automating Code Review Activities by Large-Scale Pre-training](https://mlsanity.com/paper/2203.09095/) (2022)
- [Rubric Is All You Need: Enhancing LLM-based Code Evaluation With Question-Specific Rubrics](https://mlsanity.com/paper/2503.23989/) (2025)
- [AI-TA: Towards an Intelligent Question-Answer Teaching Assistant using Open-Source LLMs](https://mlsanity.com/paper/2311.02775/) (2023)
- [Automated Grading and Feedback Tools for Programming Education: A Systematic Review](https://mlsanity.com/paper/2306.11722/) (2023)
- [Iris: An AI-Driven Virtual Tutor For Computer Science Education](https://mlsanity.com/paper/2405.08008/) (2024)
- [Towards Trustworthy AutoGrading of Short, Multi-lingual, Multi-type Answers](https://mlsanity.com/paper/2201.03425/) (2022)
- [Partnering with AI: A Pedagogical Feedback System for LLM Integration into Programming Education](https://mlsanity.com/paper/2507.00406/) (2025)
- [SPHERE: Scaling Personalized Feedback in Programming Classrooms with Structured Review of LLM Outputs](https://mlsanity.com/paper/2410.16513/) (2024)
- [SCLA: Automated Smart Contract Summarization via LLMs and Control Flow Prompt](https://mlsanity.com/paper/2402.04863/) (2024)
- [CodEv: An Automated Grading Framework Leveraging Large Language Models for Consistent and Constructive Feedback](https://mlsanity.com/paper/2501.10421/) (2025)
- [StepGrade: Grading Programming Assignments with Context-Aware LLMs](https://mlsanity.com/paper/2503.20851/) (2025)
- [An AST-guided LLM Approach for SVRF Code Synthesis](https://mlsanity.com/paper/2507.00352/) (2025)
- [Leveraging Log Probabilities in Language Models to Forecast Future Events](https://mlsanity.com/paper/2501.04880/) (2025)
- [Assessing Large Language Models for Automated Feedback Generation in Learning Programming Problem Solving](https://mlsanity.com/paper/2503.14630/) (2025)
- [On the Similarities of Embeddings in Contrastive Learning](https://mlsanity.com/paper/2506.09781/) (2025)
- [Concept-based Rubrics Improve LLM Formative Assessment and Data Synthesis](https://mlsanity.com/paper/2504.03877/) (2025)

## Abstract (arXiv)

The rapid growth of programming education has outpaced traditional assessment tools, leaving faculty with limited means to provide meaningful, scalable feedback. Conventional autograders, while efficient, act as black-box systems that simply return pass/fail results, offering little insight into student thinking or learning needs. Autograder+ is designed to shift autograding from a purely summative process to a formative learning experience. It introduces two key capabilities: automated feedback generation using a fine-tuned Large Language Model, and visualization of student code submissions to uncover learning patterns. The model is fine-tuned on curated student code and expert feedback to ensure pedagogically aligned, context-aware guidance. In evaluation across 600 student submissions from multiple programming tasks, the system produced feedback with strong semantic alignment to instructor comments. For visualization, contrastively learned code embeddings trained on 1,000 annotated submissions enable grouping solutions into meaningful clusters based on functionality and approach. The system also supports prompt-pooling, allowing instructors to guide feedback style through selected prompt templates. By integrating AI-driven feedback, semantic clustering, and interactive visualization, Autograder+ reduces instructor workload while supporting targeted instruction and promoting stronger learning outcomes.