# A Survey of Automated Programming Hint Generation -- The HINTS Framework

arXiv: https://arxiv.org/abs/1908.11566 · HTML: https://mlsanity.com/paper/1908.11566/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2019.

## Built on by (most cited first, 12 of 12)

- [CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes](https://mlsanity.com/paper/2308.06921/) (2023)
- ["It's Weird That it Knows What I Want": Usability and Interactions with Copilot for Novice Programmers](https://mlsanity.com/paper/2304.02491/) (2023)
- [Can Language Models Employ the Socratic Method? Experiments with Code Debugging](https://mlsanity.com/paper/2310.03210/) (2023)
- [TriviaHG: A Dataset for Automatic Hint Generation from Factoid Questions](https://mlsanity.com/paper/2403.18426/) (2024)
- [Navigating the Landscape of Hint Generation Research: From the Past to the Future](https://mlsanity.com/paper/2404.04728/) (2024)
- [Few-shot Question Generation for Personalized Feedback in Intelligent Tutoring Systems](https://mlsanity.com/paper/2206.04187/) (2022)
- [WikiHint: A Human-Annotated Dataset for Hint Ranking and Generation](https://mlsanity.com/paper/2412.01626/) (2024)
- [HintEval: An Open-Source Python Toolkit for Hint Generation and Hint Evaluation](https://mlsanity.com/paper/2502.00857/) (2025)
- [On the Opportunities of Large Language Models for Programming Process Data](https://mlsanity.com/paper/2411.00414/) (2024)
- [ast2vec: Utilizing Recursive Neural Encodings of Python Programs](https://mlsanity.com/paper/2103.11614/) (2021)
- [Designing and Evaluating Chain-of-Hints for Scientific Question Answering](https://mlsanity.com/paper/2510.21087/) (2025)
- [SCRIPT: Implementing an Intelligent Tutoring System for Programming in a German University Context](https://mlsanity.com/paper/2604.16117/) (2026)

## Abstract (arXiv)

Automated tutoring systems offer the flexibility and scalability necessary to facilitate the provision of high quality and universally accessible programming education. In order to realise the full potential of these systems, recent work has proposed a diverse range of techniques for automatically generating hints to assist students with programming exercises. This paper integrates these apparently disparate approaches into a coherent whole. Specifically, it emphasises that all hint techniques can be understood as a series of simpler components with similar properties. Using this insight, it presents a simple framework for describing such techniques, the Hint Iteration by Narrow-down and Transformation Steps (HINTS) framework, and it surveys recent work in the context of this framework. It discusses important implications of the survey and framework, including the need to further develop evaluation methods and the importance of considering hint technique components when designing, communicating and evaluating hint systems. Ultimately, this paper is designed to facilitate future opportunities for the development, extension and comparison of automated programming hint techniques in order to maximise their educational potential.