# You're (Not) My Type -- Can LLMs Generate Feedback of Specific Types for Introductory Programming Tasks?

arXiv: https://arxiv.org/abs/2412.03516 · HTML: https://mlsanity.com/paper/2412.03516/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 70 references that are in the map.
- First posted on arXiv 4 Dec 2024.

## Builds on (most cited first, 4 of 4)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Large Language Models in Fault Localisation](https://mlsanity.com/paper/2308.15276/) (2023)
- [How Novice Programmers Use and Experience ChatGPT when Solving Programming Exercises in an Introductory Course](https://mlsanity.com/paper/2407.20792/) (2024)
- [Large Language Models in Introductory Programming Education: ChatGPT's Performance and Implications for Assessments](https://mlsanity.com/paper/2308.08572/) (2023)

## Built on by (most cited first, 6 of 6)

- [Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Tools](https://mlsanity.com/paper/2412.14732/) (2024)
- [Prompt Programming: A Platform for Dialogue-based Computational Problem Solving with Generative AI Models](https://mlsanity.com/paper/2503.04267/) (2025)
- [Leveraging Large Language Models to Generate Course-specific Semantically Annotated Learning Objects](https://mlsanity.com/paper/2412.04185/) (2024)
- [Students' Feedback Requests and Interactions with the SCRIPT Chatbot: Do They Get What They Ask For?](https://mlsanity.com/paper/2507.17258/) (2025)
- [GenAI Voice Mode in Programming Education](https://mlsanity.com/paper/2509.10596/) (2025)
- [A Risk-Adaptive and Evidence-Constrained Framework for Generative AI Feedback in Programming Education](https://mlsanity.com/paper/2609.29874/) (2026)

## Abstract (arXiv)

Background: Feedback as one of the most influential factors for learning has been subject to a great body of research. It plays a key role in the development of educational technology systems and is traditionally rooted in deterministic feedback defined by experts and their experience. However, with the rise of generative AI and especially Large Language Models (LLMs), we expect feedback as part of learning systems to transform, especially for the context of programming. In the past, it was challenging to automate feedback for learners of programming. LLMs may create new possibilities to provide richer, and more individual feedback than ever before. Objectives: This paper aims to generate specific types of feedback for introductory programming tasks using LLMs. We revisit existing feedback taxonomies to capture the specifics of the generated feedback, such as randomness, uncertainty, and degrees of variation. Methods: We iteratively designed prompts for the generation of specific feedback types (as part of existing feedback taxonomies) in response to authentic student programs. We then evaluated the generated output and determined to what extent it reflected certain feedback types. Results and Conclusion: The present work provides a better understanding of different feedback dimensions and characteristics. The results have implications for future feedback research with regard to, for example, feedback effects and learners' informational needs. It further provides a basis for the development of new tools and learning systems for novice programmers including feedback generated by AI.