# AutoTutor meets Large Language Models: A Language Model Tutor with Rich Pedagogy and Guardrails

arXiv: https://arxiv.org/abs/2402.09216 · HTML: https://mlsanity.com/paper/2402.09216/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 35 references that are in the map.
- First posted on arXiv 14 Feb 2024.

## Builds on (most cited first, 10 of 10)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study](https://mlsanity.com/paper/2305.13860/) (2023)
- [How Language Model Hallucinations Can Snowball](https://mlsanity.com/paper/2305.13534/) (2023)
- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges](https://mlsanity.com/paper/2311.03287/) (2023)
- [CLASS: A Design Framework for building Intelligent Tutoring Systems based on Learning Science principles](https://mlsanity.com/paper/2305.13272/) (2023)
- [Generative AI for Programming Education: Benchmarking ChatGPT, GPT-4, and Human Tutors](https://mlsanity.com/paper/2306.17156/) (2023)
- [Simple Gray Coding and LLR Calculation for MDS Modulation Systems](https://mlsanity.com/paper/2201.08237/) (2022)

## Built on by (most cited first, 30 of 31)

- [Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutors](https://mlsanity.com/paper/2412.09416/) (2024)
- [Training LLM-based Tutors to Improve Student Learning Outcomes in Dialogues](https://mlsanity.com/paper/2503.06424/) (2025)
- [Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors](https://mlsanity.com/paper/2407.09136/) (2024)
- [Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure](https://mlsanity.com/paper/2410.03781/) (2024)
- [From MOOC to MAIC: Reshaping Online Teaching and Learning through LLM-driven Agents](https://mlsanity.com/paper/2409.03512/) (2024)
- [Combining Large Language Models with Tutoring System Intelligence: A Case Study in Caregiver Homework Support](https://mlsanity.com/paper/2412.11995/) (2024)
- [TutorBench: A Benchmark To Assess Tutoring Capabilities Of Large Language Models](https://mlsanity.com/paper/2510.02663/) (2025)
- [TutorGym: A Testbed for Evaluating AI Agents as Tutors and Students](https://mlsanity.com/paper/2505.01563/) (2025)
- [Multilingual Performance Biases of Large Language Models in Education](https://mlsanity.com/paper/2504.17720/) (2025)
- [Intent Matters: Enhancing AI Tutoring with Fine-Grained Pedagogical Intent Annotation](https://mlsanity.com/paper/2506.07626/) (2025)
- [Navigating the Landscape of Hint Generation Research: From the Past to the Future](https://mlsanity.com/paper/2404.04728/) (2024)
- [Position: LLMs Can be Good Tutors in English Education](https://mlsanity.com/paper/2502.05467/) (2025)
- [Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewards](https://mlsanity.com/paper/2510.01167/) (2025)
- [When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration](https://mlsanity.com/paper/2506.05579/) (2025)
- [Letting Tutor Personas Speak Up for LLMs: Learning Steering Vectors from Dialogue via Preference Optimization](https://mlsanity.com/paper/2602.07639/) (2026)
- [Awaking the Slides: A Tuning-free and Knowledge-regulated AI Tutoring System via Language Model Coordination](https://mlsanity.com/paper/2409.07372/) (2024)
- [From Solver to Tutor: Evaluating the Pedagogical Intelligence of LLMs with KMP-Bench](https://mlsanity.com/paper/2603.02775/) (2026)
- [Easy Come, Easy Go? Examining the Perceptions and Learning Effects of LLM-based Chatbot in the Context of Search-as-Learning](https://mlsanity.com/paper/2410.01396/) (2024)
- [The Imitation Game for Educational AI](https://mlsanity.com/paper/2502.15127/) (2025)
- [Designing and Evaluating Chain-of-Hints for Scientific Question Answering](https://mlsanity.com/paper/2510.21087/) (2025)
- [Large Language Models Approach Expert Pedagogical Quality in Math Tutoring but Differ in Instructional and Linguistic Profiles](https://mlsanity.com/paper/2512.20780/) (2025)
- [UnPII: Unlearning Personally Identifiable Information with Quantifiable Exposure Risk](https://mlsanity.com/paper/2601.01786/) (2026)
- [GradingAttack: Exposing Security Vulnerabilities in LLM Based Educational Grading Agents](https://mlsanity.com/paper/2602.00979/) (2026)
- [Listen, Correct, and Feed Back: Spoken Pedagogical Feedback Generation](https://mlsanity.com/paper/2604.14177/) (2026)
- [Tackling the Root of Misinformation by Teaching Laypeople about Logical Fallacies via Socratic Questioning and Critical Argumentation](https://mlsanity.com/paper/2606.01020/) (2026)
- [Lect=uraAgents: A Multi-Agent Framework for Adaptive Personalized AI-Assisted Learning and Embodied Teaching](https://mlsanity.com/paper/2606.16428/) (2026)
- [Towards Pedagogically Aligned LLM Tutors for Math Mistake Remediation](https://mlsanity.com/paper/2606.21502/) (2026)
- [Curiosity as Linguistic Intervention: Using LLM Tutoring Dialogues to Influence Exploratory Learning Behavior](https://mlsanity.com/paper/2606.22349/) (2026)
- [PIVOT: Preference-based Intervention Vectors for Pedagogical Tutor Steering](https://mlsanity.com/paper/2608.07509/) (2026)
- [From Content Generation to Learning Support: Pedagogy-Guided Generative Video Tutors for STEM Learning](https://mlsanity.com/paper/2609.24083/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have found several use cases in education, ranging from automatic question generation to essay evaluation. In this paper, we explore the potential of using Large Language Models (LLMs) to author Intelligent Tutoring Systems. A common pitfall of LLMs is their straying from desired pedagogical strategies such as leaking the answer to the student, and in general, providing no guarantees. We posit that while LLMs with certain guardrails can take the place of subject experts, the overall pedagogical design still needs to be handcrafted for the best learning results. Based on this principle, we create a sample end-to-end tutoring system named MWPTutor, which uses LLMs to fill in the state space of a pre-defined finite state transducer. This approach retains the structure and the pedagogy of traditional tutoring systems that has been developed over the years by learning scientists but brings in additional flexibility of LLM-based approaches. Through a human evaluation study on two datasets based on math word problems, we show that our hybrid approach achieves a better overall tutoring score than an instructed, but otherwise free-form, GPT-4. MWPTutor is completely modular and opens up the scope for the community to improve its performance by improving individual modules or using different teaching strategies that it can follow.