# Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakes

arXiv: https://arxiv.org/abs/2310.10648 · HTML: https://mlsanity.com/paper/2310.10648/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 87 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 11 of 11)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialogues](https://mlsanity.com/paper/2205.07540/) (2022)
- [Cognitive Reframing of Negative Thoughts through Human-Language Model Interaction](https://mlsanity.com/paper/2305.02466/) (2023)
- [GODEL: Large-Scale Pre-Training for Goal-Directed Dialog](https://mlsanity.com/paper/2206.11309/) (2022)
- [Is ChatGPT a Good Teacher Coach? Measuring Zero-Shot Performance For Scoring and Providing Actionable Insights on Classroom Instruction](https://mlsanity.com/paper/2306.03090/) (2023)
- [Opportunities and Challenges in Neural Dialog Tutoring](https://mlsanity.com/paper/2301.09919/) (2023)
- [Too many cooks: Bayesian inference for coordinating multi-agent collaboration](https://mlsanity.com/paper/2003.11778/) (2020)
- [SIGHT: A Large Annotated Dataset on Student Insights Gathered from Higher Education Transcripts](https://mlsanity.com/paper/2306.09343/) (2023)
- [Eliciting Knowledge from Experts:Automatic Transcript Parsing for Cognitive Task Analysis](https://mlsanity.com/paper/1906.11384/) (2019)

## Built on by (most cited first, 30 of 52)

- [Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutors](https://mlsanity.com/paper/2412.09416/) (2024)
- [Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise](https://mlsanity.com/paper/2410.03017/) (2024)
- [MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors](https://mlsanity.com/paper/2502.18940/) (2025)
- [Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMs](https://mlsanity.com/paper/2409.16490/) (2024)
- [Training LLM-based Tutors to Improve Student Learning Outcomes in Dialogues](https://mlsanity.com/paper/2503.06424/) (2025)
- [Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors](https://mlsanity.com/paper/2407.09136/) (2024)
- [Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure](https://mlsanity.com/paper/2410.03781/) (2024)
- [Training Turn-by-Turn Verifiers for Dialogue Tutoring Agents: The Curious Case of LLMs as Your Coding Tutors](https://mlsanity.com/paper/2502.13311/) (2025)
- [Embracing Imperfection: Simulating Students with Diverse Cognitive Levels Using LLM-based Agents](https://mlsanity.com/paper/2505.19997/) (2025)
- [Findings of the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutors](https://mlsanity.com/paper/2507.10579/) (2025)
- [From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning](https://mlsanity.com/paper/2505.15607/) (2025)
- [Instruct, Not Assist: LLM-based Multi-Turn Planning and Hierarchical Questioning for Socratic Code Debugging](https://mlsanity.com/paper/2406.11709/) (2024)
- [Contrastive Explanations That Anticipate Human Misconceptions Can Improve Human Decision-Making Skills](https://mlsanity.com/paper/2410.04253/) (2024)
- [The Call for Socially Aware Language Technologies](https://mlsanity.com/paper/2405.02411/) (2024)
- [Toward In-Context Teaching: Adapting Examples to Students' Misconceptions](https://mlsanity.com/paper/2405.04495/) (2024)
- [PIIvot: A Lightweight NLP Anonymization Framework for Question-Anchored Tutoring Dialogues](https://mlsanity.com/paper/2505.16931/) (2025)
- [Can Vision-Language Models Evaluate Handwritten Math?](https://mlsanity.com/paper/2501.07244/) (2025)
- [Automatic Generation of Question Hints for Mathematics Problems using Large Language Models in Educational Technology](https://mlsanity.com/paper/2411.03495/) (2024)
- [What do Large Language Models Say About Animals? Investigating Risks of Animal Harm in Generated Text](https://mlsanity.com/paper/2503.04804/) (2025)
- [Intent Matters: Enhancing AI Tutoring with Fine-Grained Pedagogical Intent Annotation](https://mlsanity.com/paper/2506.07626/) (2025)
- [LLMs in Education: Novel Perspectives, Challenges, and Opportunities](https://mlsanity.com/paper/2409.11917/) (2024)
- [Evaluating Answer Leakage Robustness of LLM Tutors against Adversarial Student Attacks](https://mlsanity.com/paper/2604.18660/) (2026)
- [Tree-of-Debate: Multi-Persona Debate Trees Elicit Critical Thinking for Scientific Comparative Analysis](https://mlsanity.com/paper/2502.14767/) (2025)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Discerning minds or generic tutors? Evaluating instructional guidance capabilities in Socratic LLMs](https://mlsanity.com/paper/2508.06583/) (2025)
- [Pedagogy-driven Evaluation of Generative AI-powered Intelligent Tutoring Systems](https://mlsanity.com/paper/2510.22581/) (2025)
- [MSA at BEA 2025 Shared Task: Disagreement-Aware Instruction Tuning for Multi-Dimensional Evaluation of LLMs as Math Tutors](https://mlsanity.com/paper/2505.18549/) (2025)
- [When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration](https://mlsanity.com/paper/2506.05579/) (2025)
- [Exploring LLMs for Predicting Tutor Strategy and Student Outcomes in Dialogues](https://mlsanity.com/paper/2507.06910/) (2025)
- [Simulating Student-Tutor Interactions with LLMs for Multilingual Math Feedback](https://mlsanity.com/paper/2506.04920/) (2025)

## Abstract (arXiv)

Scaling high-quality tutoring remains a major challenge in education. Due to growing demand, many platforms employ novice tutors who, unlike experienced educators, struggle to address student mistakes and thus fail to seize prime learning opportunities. Our work explores the potential of large language models (LLMs) to close the novice-expert knowledge gap in remediating math mistakes. We contribute Bridge, a method that uses cognitive task analysis to translate an expert's latent thought process into a decision-making model for remediation. This involves an expert identifying (A) the student's error, (B) a remediation strategy, and (C) their intention before generating a response. We construct a dataset of 700 real tutoring conversations, annotated by experts with their decisions. We evaluate state-of-the-art LLMs on our dataset and find that the expert's decision-making model is critical for LLMs to close the gap: responses from GPT4 with expert decisions (e.g., "simplify the problem") are +76% more preferred than without. Additionally, context-sensitive decisions are critical to closing pedagogical gaps: random decisions decrease GPT4's response quality by -97% than expert decisions. Our work shows the potential of embedding expert thought processes in LLM generations to enhance their capability to bridge novice-expert knowledge gaps. Our dataset and code can be found at: urlhttps://github.com/rosewang2008/bridge.