# How well do Large Language Models Recognize Instructional Moves? Establishing Baselines for Foundation Models in Educational Discourse

arXiv: https://arxiv.org/abs/2512.19903 · HTML: https://mlsanity.com/paper/2512.19903/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 22 Dec 2025.

## Builds on (most cited first, 12 of 12)

- [Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise](https://mlsanity.com/paper/2410.03017/) (2024)
- [MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors](https://mlsanity.com/paper/2502.18940/) (2025)
- [The TalkMoves Dataset: K-12 Mathematics Lesson Transcripts Annotated for Teacher and Student Discursive Moves](https://mlsanity.com/paper/2204.09652/) (2022)
- [If in a Crowdsourced Data Annotation Pipeline, a GPT-4](https://mlsanity.com/paper/2402.16795/) (2024)
- [Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance](https://mlsanity.com/paper/2410.18889/) (2024)
- [Evaluating Large Language Models in Analysing Classroom Dialogue](https://mlsanity.com/paper/2402.02380/) (2024)
- [Prompting in the Dark: Assessing Human Performance in Prompt Engineering for Data Labeling When Gold Labels Are Absent](https://mlsanity.com/paper/2502.11267/) (2025)
- [Benchmarking Generative AI for Scoring Medical Student Interviews in Objective Structured Clinical Examinations (OSCEs)](https://mlsanity.com/paper/2501.13957/) (2025)
- [A Benchmark for Math Misconceptions: Bridging Gaps in Middle School Algebra with AI-Supported Instruction](https://mlsanity.com/paper/2412.03765/) (2024)
- [Benchmarking the Pedagogical Knowledge of Large Language Models](https://mlsanity.com/paper/2506.18710/) (2025)
- [Leveraging Human Feedback to Scale Educational Datasets: Combining Crowdworkers and Comparative Judgement](https://mlsanity.com/paper/2305.12894/) (2023)
- [MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring](https://mlsanity.com/paper/2510.23477/) (2025)

## Built on by (most cited first, 2 of 2)

- [LLM Reasoning Predicts When Models Are Right: Evidence from Coding Classroom Discourse](https://mlsanity.com/paper/2602.09832/) (2026)
- [EduBehaviors: Assertion-based Schemas for Auditable Coding of Educational Dialogues](https://mlsanity.com/paper/2609.27043/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly adopted in educational technologies for a variety of tasks, from generating instructional materials and assisting with assessment design to tutoring. While prior work has investigated how models can be adapted or optimized for specific tasks, far less is known about how well LLMs perform at interpreting authentic educational scenarios without significant customization. As LLM-based systems become widely adopted by learners and educators in everyday academic contexts, understanding their out-of-the-box capabilities is increasingly important for setting expectations and benchmarking. We compared six LLMs to estimate their baseline performance on a simple but important task: classifying instructional moves in authentic classroom transcripts. We evaluated typical prompting methods: zero-shot, one-shot, and few-shot prompting. We found that while zero-shot performance was moderate, providing comprehensive examples (few-shot prompting) significantly improved performance for state-of-the-art models, with the strongest configuration reaching Cohen's Kappa = 0.58 against expert-coded annotations. At the same time, improvements were neither uniform nor complete: performance varied considerably by instructional move, and higher recall frequently came at the cost of increased false positives. Overall, these findings indicate that foundation models demonstrate meaningful yet limited capacity to interpret instructional discourse, with prompt design helping to surface capability but not eliminating fundamental reliability constraints.