# Understanding Student Interaction with AI-Powered Next-Step Hints: Strategies and Challenges

arXiv: https://arxiv.org/abs/2511.06362 · HTML: https://mlsanity.com/paper/2511.06362/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 28 references that are in the map.
- First posted on arXiv 9 Nov 2025.

## Builds on (most cited first, 9 of 9)

- [CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes](https://mlsanity.com/paper/2308.06921/) (2023)
- [Using Large Language Models to Enhance Programming Error Messages](https://mlsanity.com/paper/2210.11630/) (2022)
- [Next-Step Hint Generation for Introductory Programming Using Large Language Models](https://mlsanity.com/paper/2312.10055/) (2023)
- [Exploring How Multiple Levels of GPT-Generated Programming Hints Support or Disappoint Novices](https://mlsanity.com/paper/2404.02213/) (2024)
- [Bridging Education and Development: IDEs as Interactive Learning Platforms](https://mlsanity.com/paper/2401.14284/) (2024)
- [Integrating ChatGPT in a Computer Science Course: Students Perceptions and Suggestions](https://mlsanity.com/paper/2402.01640/) (2023)
- [One Step at a Time: Combining LLMs and Static Analysis to Generate Next-Step Hints for Programming Tasks](https://mlsanity.com/paper/2410.09268/) (2024)
- [KOALA: a Configurable Tool for Collecting IDE Data When Solving Programming Tasks](https://mlsanity.com/paper/2506.21266/) (2025)
- [Hyperstyle: A Tool for Assessing the Code Quality of Solutions to Programming Assignments](https://mlsanity.com/paper/2112.02963/) (2021)

## Abstract (arXiv)

Automated feedback generation plays a crucial role in enhancing personalized learning experiences in computer science education. Among different types of feedback, next-step hint feedback is particularly important, as it provides students with actionable steps to progress towards solving programming tasks. This study investigates how students interact with an AI-driven next-step hint system in an in-IDE learning environment. We gathered and analyzed a dataset from 34 students solving Kotlin tasks, containing detailed hint interaction logs. We applied process mining techniques and identified 16 common interaction scenarios. Semi-structured interviews with 6 students revealed strategies for managing unhelpful hints, such as adapting partial hints or modifying code to generate variations of the same hint. These findings, combined with our publicly available dataset, offer valuable opportunities for future research and provide key insights into student behavior, helping improve hint design for enhanced learning support.