# AI-Tutoring in Software Engineering Education

arXiv: https://arxiv.org/abs/2404.02548 · HTML: https://mlsanity.com/paper/2404.02548/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 47 references that are in the map.
- First posted on arXiv 3 Apr 2024.

## Builds on (most cited first, 5 of 5)

- [Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study](https://mlsanity.com/paper/2305.13860/) (2023)
- [An Analysis of the Automatic Bug Fixing Performance of ChatGPT](https://mlsanity.com/paper/2301.08653/) (2023)
- [Is ChatGPT the Ultimate Programming Assistant -- How far is it?](https://mlsanity.com/paper/2304.11938/) (2023)
- [Learning gain differences between ChatGPT and human tutor generated algebra hints](https://mlsanity.com/paper/2302.06871/) (2023)
- [ChatGPT and Software Testing Education: Promises & Perils](https://mlsanity.com/paper/2302.03287/) (2023)

## Built on by (most cited first, 11 of 11)

- [Large Language Models in Computer Science Education: A Systematic Literature Review](https://mlsanity.com/paper/2410.16349/) (2024)
- [The Impact of LLM-Assistants on Software Developer Productivity: A Systematic Review and Mapping Study](https://mlsanity.com/paper/2507.03156/) (2025)
- [Assessing Personalized AI Mentoring with Large Language Models in the Computing Field](https://mlsanity.com/paper/2412.08430/) (2024)
- [DeepTutor: Towards Agentic Personalized Tutoring](https://mlsanity.com/paper/2604.26962/) (2026)
- [A Survey Study on the State of the Art of Programming Exercise Generation using Large Language Models](https://mlsanity.com/paper/2405.20183/) (2024)
- [From Junior to Senior: Allocating Agency and Navigating Professional Growth in Agentic AI-Mediated Software Engineering](https://mlsanity.com/paper/2602.00496/) (2026)
- [LLMs Integration in Software Engineering Team Projects: Roles, Impact, and a Pedagogical Design Space for AI Tools in Computing Education](https://mlsanity.com/paper/2410.23069/) (2024)
- [An Experience Report on a Pedagogically Controlled, Curriculum-Constrained AI Tutor for SE Education](https://mlsanity.com/paper/2512.11882/) (2025)
- [From Domain Understanding to Design Readiness: a playbook for GenAI-supported learning in Software Engineering](https://mlsanity.com/paper/2604.00120/) (2026)
- [Chatbot-Based Assessment of Code Understanding in Automated Programming Assessment Systems](https://mlsanity.com/paper/2604.07304/) (2026)
- [Pedagogical Promise and Peril of AI: A Text Mining Analysis of ChatGPT Research Discussions in Programming Education](https://mlsanity.com/paper/2605.00361/) (2026)

## Abstract (arXiv)

With the rapid advancement of artificial intelligence (AI) in various domains, the education sector is set for transformation. The potential of AI-driven tools in enhancing the learning experience, especially in programming, is immense. However, the scientific evaluation of Large Language Models (LLMs) used in Automated Programming Assessment Systems (APASs) as an AI-Tutor remains largely unexplored. Therefore, there is a need to understand how students interact with such AI-Tutors and to analyze their experiences. In this paper, we conducted an exploratory case study by integrating the GPT-3.5-Turbo model as an AI-Tutor within the APAS Artemis. Through a combination of empirical data collection and an exploratory survey, we identified different user types based on their interaction patterns with the AI-Tutor. Additionally, the findings highlight advantages, such as timely feedback and scalability. However, challenges like generic responses and students' concerns about a learning progress inhibition when using the AI-Tutor were also evident. This research adds to the discourse on AI's role in education.