mlsanityData to 14 Oct 2026

MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems

arXiv 2305.14536 ↗First posted 23 May 2023cs.CLcited by 131

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

While automatic dialogue tutors hold great potential in making education personalized and more accessible, research on such systems has been hampered by a lack of sufficiently large and high-quality datasets. Collecting such datasets remains challenging, as recording tutoring sessions raises privacy concerns and crowdsourcing leads to insufficient data quality. To address this, we propose a framework to generate such dialogues by pairing human teachers with a Large Language Model (LLM) prompted to represent common student errors. We describe how we use this framework to collect MathDial, a dataset of 3k one-to-one teacher-student tutoring dialogues grounded in multi-step math reasoning problems. While models like GPT-3 are good problem solvers, they fail at tutoring because they generate factually incorrect feedback or are prone to revealing solutions to students too early. To overcome this, we let teachers provide learning opportunities to students by guiding them using various scaffolding questions according to a taxonomy of teacher moves. We demonstrate MathDial and its extensive annotations can be used to finetune models to be more effective tutors (and not just solvers). We confirm this by automatic and human evaluation, notably in an interactive setting that measures the trade-off between student solving success and telling solutions. The dataset is released publicly.

Key facts. Cited by 131 papers in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 46 of its 80 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

46 of 80 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  9. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  12. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
34 more
  1. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  2. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 652
  3. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  4. 2019Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetcited by 434
  5. 2022Using cognitive psychology to understand GPT-3cited by 398
  6. 2018QuAC : Question Answering in Contextcited by 381
  7. 2023Mathematical Capabilities of ChatGPTcited by 268
  8. 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
  9. 2023Can Large Language Models Transform Computational Social Science?cited by 250
  10. 2020MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselinescited by 204
  11. 2022Language models show human-like content effects on reasoning taskscited by 173
  12. 2022SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualizationcited by 168
  13. 2019Taskmaster-1: Toward a Realistic and Diverse Dialog Datasetcited by 140
  14. 2018Generating Informative and Diverse Conversational Responses via Adversarial Information Maximizationcited by 116
  15. 2019Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systemscited by 110
  16. 2022FaithDial: A Faithful Benchmark for Information-Seeking Dialoguecited by 109
  17. 2021MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluationcited by 88
  18. 2022The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialoguescited by 79
  19. 2023PLACES: Prompting Language Models for Social Conversation Synthesiscited by 72
  20. 2018Building a Conversational Agent Overnight with Dialogue Self-Playcited by 70
  21. 2020CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Datasetcited by 66
  22. 2020Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Accesscited by 58
  23. 2022Dialog Inpainting: Turning Documents into Dialogscited by 56
  24. 2020MultiWOZ 2.3: A multi-domain task-oriented dialogue dataset enhanced with annotation corrections and co-reference annotationcited by 51
  25. 2021Measuring Conversational Uptake: A Case Study on Student-Teacher Interactionscited by 44
  26. 2022The NCTE Transcripts: A Dataset of Elementary Math Classroom Transcriptscited by 43
  27. 2022Automatic Generation of Socratic Subquestions for Teaching Math Word Problemscited by 43
  28. 2023Opportunities and Challenges in Neural Dialog Tutoringcited by 33
  29. 2023Elastic Weight Removal for Faithful and Abstractive Dialogue Generationcited by 29
  30. 2018Augmenting Neural Response Generation with Context-Aware Topical Attentioncited by 27
  31. 2022Cross-Lingual Dialogue Dataset Creation via Outline-Based Generationcited by 23
  32. 2020The Teacher-Student Chatroom Corpuscited by 17
  33. 2023Strategize Before Teaching: A Conversational Tutoring System with Pedagogy Self-Distillationcited by 11
  34. 2022Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarizationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 23; 2025: 48; 2026: 55.

131 papers.

Sort:

By area:

  1. 2024Can Language Models Solve Olympiad Programming?cited by 72
  2. 2024Unifying AI Tutor Evaluation: An Evaluation Taxonomy for Pedagogical Ability Assessment of LLM-Powered AI Tutorscited by 61
  3. 2025LLM Agents for Education: Advances and Applicationscited by 59
  4. 2023Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakescited by 52
  5. 2024Towards Human-centered Proactive Conversational Agentscited by 49
  6. 2025MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutorscited by 49
  7. 2024Designing a Dashboard for Transparency and Control of Conversational AIcited by 46
  8. 2024Exploring Knowledge Tracing in Tutor-Student Dialogues using LLMscited by 46
  9. 2025Training LLM-based Tutors to Improve Student Learning Outcomes in Dialoguescited by 43
  10. 2024Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutorscited by 41
  11. 2024MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Educationcited by 37
  12. 2023Exploring the Numerical Reasoning Capabilities of Language Models: A Comprehensive Analysis on Tabular Datacited by 33
38 more
  1. 2023MathAttack: Attacking Large Language Models Towards Math Solving Abilitycited by 32
  2. 2024AutoTutor meets Large Language Models: A Language Model Tutor with Rich Pedagogy and Guardrailscited by 31
  3. 2024Book2Dial: Generating Teacher-Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbotscited by 25
  4. 2023Mathematical Language Models: A Surveycited by 24
  5. 2024Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failurecited by 24
  6. 2025Training Turn-by-Turn Verifiers for Dialogue Tutoring Agents: The Curious Case of LLMs as Your Coding Tutorscited by 20
  7. 2025Learning to Make MISTAKEs: Modeling Incorrect Student Thinking And Key Errorscited by 20
  8. 2024Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?cited by 19
  9. 2025Findings of the BEA 2025 Shared Task on Pedagogical Ability Assessment of AI-powered Tutorscited by 18
  10. 2026Simulated Students in Tutoring Dialogues: Substance or Illusion?cited by 18
  11. 2024Language Models as Science Tutorscited by 17
  12. 2025One Size doesn't Fit All: A Personalized Conversational Tutoring Agent for Mathematics Instructioncited by 17
  13. 2025From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learningcited by 17
  14. 2024Boosting Large Language Models with Socratic Method for Conversational Mathematics Teachingcited by 16
  15. 2024LLM Roleplay: Simulating Human-Chatbot Interactioncited by 15
  16. 2024Personality-aware Student Simulation for Conversational Intelligent Tutoring Systemscited by 12
  17. 2025A Survey on Mathematical Reasoning and Optimization with Large Language Modelscited by 12
  18. 2023TPE: Towards Better Compositional Reasoning over Conceptual Tools with Multi-persona Collaborationcited by 11
  19. 2025PIIvot: A Lightweight NLP Anonymization Framework for Question-Anchored Tutoring Dialoguescited by 11
  20. 2025Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimizationcited by 10
  21. 2025The StudyChat Dataset: Analyzing Student Dialogues With ChatGPT in an Artificial Intelligence Coursecited by 9
  22. 2025TeachLM: Post-Training LLMs for Education Using Authentic Learning Datacited by 9
  23. 2024A Systematic Review on Prompt Engineering in Large Language Models for K-12 STEM Educationcited by 8
  24. 2025Do LLMs Make Mistakes Like Students? Exploring Natural Alignment between Language Models and Human Error Patternscited by 8
  25. 2025Intent Matters: Enhancing AI Tutoring with Fine-Grained Pedagogical Intent Annotationcited by 8
  26. 2024LLMs in Education: Novel Perspectives, Challenges, and Opportunitiescited by 7
  27. 2025LLMs cannot spot math errors, even when allowed to peek into the solutioncited by 7
  28. 2026Evaluating Answer Leakage Robustness of LLM Tutors against Adversarial Student Attackscited by 7
  29. 2024Navigating the Landscape of Hint Generation Research: From the Past to the Futurecited by 6
  30. 2025AI-Powered Math Tutoring: Platform for Personalized and Adaptive Educationcited by 6
  31. 2025Opportunities and Challenges of LLMs in Education: An NLP Perspectivecited by 6
  32. 2025Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewardscited by 6
  33. 2024Developing a Tutoring Dialog Dataset to Optimize LLMs for Educational Usecited by 5
  34. 2025KMI: A Dataset of Korean Motivational Interviewing Dialogues for Psychotherapycited by 5
  35. 2025Discerning minds or generic tutors? Evaluating instructional guidance capabilities in Socratic LLMscited by 5
  36. 2025Pedagogy-driven Evaluation of Generative AI-powered Intelligent Tutoring Systemscited by 5
  37. 2024Battling Botpoop using GenAI for Higher Education: A Study of a Retrieval Augmented Generation Chatbots Impact on Learningcited by 4
  38. 2025MSA at BEA 2025 Shared Task: Disagreement-Aware Instruction Tuning for Multi-Dimensional Evaluation of LLMs as Math Tutorscited by 4

50 most cited of 131 All 131 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024MathChat: Benchmarking Mathematical Reasoning and Instruction Following in Multi-Turn Interactions
  2. 2025Toward Automated Qualitative Analysis: Leveraging Large Language Models for Tutoring Dialogue Evaluation
  3. 2024Student Data Paradox and Curious Case of Single Student-Tutor Model: Regressive Side Effects of Training LLMs for Personalized Learning
  4. 2024MIND: Math Informed syNthetic Dialogues for Pretraining LLMs
  5. 2024Can Language Models Rival Mathematics Students? Evaluating Mathematical Reasoning through Textual Manipulation and Human Experiments
  6. 2024MathScale: Scaling Instruction Tuning for Mathematical Reasoning
  7. 2026The Tutoring Effectiveness Index: Predicting LLM Math Tutor Quality from Four Conversation Signals
  8. 2025MMTutorBench: The First Multimodal Benchmark for AI Math Tutoring