# Tutorly: Turning Programming Videos Into Apprenticeship Learning Environments with LLMs

arXiv: https://arxiv.org/abs/2405.12946 · HTML: https://mlsanity.com/paper/2405.12946/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 105 references that are in the map.
- First posted on arXiv 21 May 2024.

## Builds on (most cited first, 8 of 8)

- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliency](https://mlsanity.com/paper/2208.06773/) (2022)
- [Interactive Task and Concept Learning from Natural Language Instructions and GUI Demonstrations](https://mlsanity.com/paper/1909.00031/) (2019)
- [Video Question Answering on Screencast Tutorials](https://mlsanity.com/paper/2008.00544/) (2020)
- [Soloist: Generating Mixed-Initiative Tutorials from Existing Guitar Instructional Videos Through Audio Processing](https://mlsanity.com/paper/2101.08846/) (2021)
- [Towards Supporting Programming Education at Scale via Live Streaming](https://mlsanity.com/paper/2010.15015/) (2020)
- [ITSS: Interactive Web-Based Authoring and Playback Integrated Environment for Programming Tutorials](https://mlsanity.com/paper/2204.08593/) (2022)

## Built on by (most cited first, 3 of 3)

- [ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inference](https://mlsanity.com/paper/2509.14537/) (2025)
- [Toward Personalizing Quantum Computing Education: An Evolutionary LLM-Powered Approach](https://mlsanity.com/paper/2504.18603/) (2025)
- [Adaptive Learning Systems: Personalized Curriculum Design Using LLM-Powered Analytics](https://mlsanity.com/paper/2507.18949/) (2025)

## Abstract (arXiv)

Online programming videos, including tutorials and streamcasts, are widely popular and contain a wealth of expert knowledge. However, effectively utilizing these resources to achieve targeted learning goals can be challenging. Unlike direct tutoring, video content lacks tailored guidance based on individual learning paces, personalized feedback, and interactive engagement necessary for support and monitoring. Our work transforms programming videos into one-on-one tutoring experiences using the cognitive apprenticeship framework. Tutorly, developed as a JupyterLab Plugin, allows learners to (1) set personalized learning goals, (2) engage in learning-by-doing through a conversational LLM-based mentor agent, (3) receive guidance and feedback based on a student model that steers the mentor moves. In a within-subject study with 16 participants learning exploratory data analysis from a streamcast, Tutorly significantly improved their performance from 61.9% to 76.6% based on a post-test questionnaire. Tutorly demonstrates the potential for enhancing programming video learning experiences with LLM and learner modeling.