# Patterns of Student Help-Seeking When Using a Large Language Model-Powered Programming Assistant

arXiv: https://arxiv.org/abs/2310.16984 · HTML: https://mlsanity.com/paper/2310.16984/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Oct 2023.

## Built on by (most cited first, 13 of 13)

- [A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education](https://mlsanity.com/paper/2312.03173/) (2023)
- [Instruct, Not Assist: LLM-based Multi-Turn Planning and Hierarchical Questioning for Socratic Code Debugging](https://mlsanity.com/paper/2406.11709/) (2024)
- [Beyond the Hype: A Comprehensive Review of Current Trends in Generative AI Research, Teaching Practices, and Tools](https://mlsanity.com/paper/2412.14732/) (2024)
- [Large Language Models in Computer Science Education: A Systematic Literature Review](https://mlsanity.com/paper/2410.16349/) (2024)
- [CourseAssist: Pedagogically Appropriate AI Tutor for Computer Science Education](https://mlsanity.com/paper/2407.10246/) (2024)
- [Efficient Classification of Student Help Requests in Programming Courses Using Large Language Models](https://mlsanity.com/paper/2310.20105/) (2023)
- [Survey of Natural Language Processing for Education: Taxonomy, Systematic Review, and Future Trends](https://mlsanity.com/paper/2401.07518/) (2024)
- [Generating Feedback-Ladders for Logical Errors in Programming using Large Language Models](https://mlsanity.com/paper/2405.00302/) (2024)
- ["All Roads Lead to ChatGPT": How Generative AI is Eroding Social Interactions and Student Learning Communities](https://mlsanity.com/paper/2504.09779/) (2025)
- [JELAI: Integrating AI and Learning Analytics in Jupyter Notebooks](https://mlsanity.com/paper/2505.17593/) (2025)
- [AI-Assisted Help-Seeking Trajectories in Programming Education from an SRL-Informed Perspective](https://mlsanity.com/paper/2606.22809/) (2026)
- [Guardrails or Roadblocks? Effects of Pedagogical Style and Context Awareness in AI Teaching Assistants for Programming](https://mlsanity.com/paper/2609.29995/) (2026)
- [Examining Variation in How Guided AI Tutors Resolve Student Impasses](https://mlsanity.com/paper/2609.38346/) (2026)

## Abstract (arXiv)

Providing personalized assistance at scale is a long-standing challenge for computing educators, but a new generation of tools powered by large language models (LLMs) offers immense promise. Such tools can, in theory, provide on-demand help in large class settings and be configured with appropriate guardrails to prevent misuse and mitigate common concerns around learner over-reliance. However, the deployment of LLM-powered tools in authentic classroom settings is still rare, and very little is currently known about how students will use them in practice and what type of help they will seek. To address this, we examine students' use of an innovative LLM-powered tool that provides on-demand programming assistance without revealing solutions directly. We deployed the tool for 12 weeks in an introductory computer and data science course (n = 52), collecting more than 2,500 queries submitted by students throughout the term. We manually categorized all student queries based on the type of assistance sought, and we automatically analyzed several additional query characteristics. We found that most queries requested immediate help with programming assignments, whereas fewer requests asked for help on related concepts or for deepening conceptual understanding. Furthermore, students often provided minimal information to the tool, suggesting this is an area in which targeted instruction would be beneficial. We also found that students who achieved more success in the course tended to have used the tool more frequently overall. Lessons from this research can be leveraged by programming educators and institutions who plan to augment their teaching with emerging LLM-powered tools.