# Opportunities and Challenges in Neural Dialog Tutoring

arXiv: https://arxiv.org/abs/2301.09919 · HTML: https://mlsanity.com/paper/2301.09919/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 70 references that are in the map.
- First posted on arXiv 24 Jan 2023.

## Builds on (most cited first, 30 of 40)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [Internet-Augmented Dialogue Generation](https://mlsanity.com/paper/2107.07566/) (2021)
- [BlenderBot 3: a deployed conversational agent that continually learns to responsibly engage](https://mlsanity.com/paper/2208.03188/) (2022)
- [USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generation](https://mlsanity.com/paper/2005.00456/) (2020)
- [The Second Conversational Intelligence Challenge (ConvAI2)](https://mlsanity.com/paper/1902.00098/) (2019)
- [COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics](https://mlsanity.com/paper/2202.11705/) (2022)
- [ParlAI: A Dialog Research Software Platform](https://mlsanity.com/paper/1705.06476/) (2017)
- [Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering](https://mlsanity.com/paper/2104.08202/) (2021)
- [Training Millions of Personalized Dialogue Agents](https://mlsanity.com/paper/1809.01984/) (2018)
- [A Dataset for Document Grounded Conversations](https://mlsanity.com/paper/1809.07358/) (2018)
- [GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection](https://mlsanity.com/paper/2111.14592/) (2021)
- [doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset](https://mlsanity.com/paper/2011.06623/) (2020)
- [PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning](https://mlsanity.com/paper/2006.16779/) (2020)

## Built on by (most cited first, 30 of 33)

- [MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems](https://mlsanity.com/paper/2305.14536/) (2023)
- [Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents](https://mlsanity.com/paper/2311.00262/) (2023)
- [Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakes](https://mlsanity.com/paper/2310.10648/) (2023)
- [MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutors](https://mlsanity.com/paper/2502.18940/) (2025)
- [Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors](https://mlsanity.com/paper/2407.09136/) (2024)
- [Book2Dial: Generating Teacher-Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbots](https://mlsanity.com/paper/2403.03307/) (2024)
- [Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failure](https://mlsanity.com/paper/2410.03781/) (2024)
- [LLM-powered Multi-agent Framework for Goal-oriented Learning in Intelligent Tutoring System](https://mlsanity.com/paper/2501.15749/) (2025)
- [From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learning](https://mlsanity.com/paper/2505.15607/) (2025)
- [Scaffolding Language Learning via Multi-modal Tutoring Systems with Pedagogical Instructions](https://mlsanity.com/paper/2404.03429/) (2024)
- [Personality-aware Student Simulation for Conversational Intelligent Tutoring Systems](https://mlsanity.com/paper/2404.06762/) (2024)
- [Strategize Before Teaching: A Conversational Tutoring System with Pedagogy Self-Distillation](https://mlsanity.com/paper/2302.13496/) (2023)
- [TPE: Towards Better Compositional Reasoning over Conceptual Tools with Multi-persona Collaboration](https://mlsanity.com/paper/2309.16090/) (2023)
- [PIIvot: A Lightweight NLP Anonymization Framework for Question-Anchored Tutoring Dialogues](https://mlsanity.com/paper/2505.16931/) (2025)
- [Plan-Grounded Large Language Models for Dual Goal Conversational Settings](https://mlsanity.com/paper/2402.01053/) (2024)
- [Towards Goal-oriented Intelligent Tutoring Systems in Online Education](https://mlsanity.com/paper/2312.10053/) (2023)
- [BIPED: Pedagogically Informed Tutoring System for ESL Education](https://mlsanity.com/paper/2406.03486/) (2024)
- [AI-Powered Math Tutoring: Platform for Personalized and Adaptive Education](https://mlsanity.com/paper/2507.12484/) (2025)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Developing a Tutoring Dialog Dataset to Optimize LLMs for Educational Use](https://mlsanity.com/paper/2410.19231/) (2024)
- [Pedagogy-driven Evaluation of Generative AI-powered Intelligent Tutoring Systems](https://mlsanity.com/paper/2510.22581/) (2025)
- [ComperDial: Commonsense Persona-grounded Dialogue Dataset and Benchmark](https://mlsanity.com/paper/2406.11228/) (2024)
- [A Framework for Situating Innovations, Opportunities, and Challenges in Advancing Vertical Systems with Large AI Models](https://mlsanity.com/paper/2504.02793/) (2025)
- [SingaKids: A Multilingual Multimodal Dialogic Tutor for Language Learning](https://mlsanity.com/paper/2506.02412/) (2025)
- [Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact](https://mlsanity.com/paper/2603.00883/) (2026)
- [RAM2C: A Liberal Arts Educational Chatbot based on Retrieval-augmented Multi-role Multi-expert Collaboration](https://mlsanity.com/paper/2409.15461/) (2024)
- [AITutor-EvalKit: Exploring the Capabilities of AI Tutors](https://mlsanity.com/paper/2512.03688/) (2025)
- [ConvoLearn: A Learning Sciences Grounded Dataset for Fine-Tuning Dialogic AI Tutors](https://mlsanity.com/paper/2601.08950/) (2026)
- [The Missing Evaluation Axis: What 10,000 Student Submissions Reveal About AI Tutor Effectiveness](https://mlsanity.com/paper/2605.05648/) (2026)
- [Uncertainty-Aware Generation and Decision-Making Under Ambiguity](https://mlsanity.com/paper/2606.30578/) (2026)

## Abstract (arXiv)

Designing dialog tutors has been challenging as it involves modeling the diverse and complex pedagogical strategies employed by human tutors. Although there have been significant recent advances in neural conversational systems using large language models (LLMs) and growth in available dialog corpora, dialog tutoring has largely remained unaffected by these advances. In this paper, we rigorously analyze various generative language models on two dialog tutoring datasets for language learning using automatic and human evaluations to understand the new opportunities brought by these advances as well as the challenges we must overcome to build models that would be usable in real educational settings. We find that although current approaches can model tutoring in constrained learning scenarios when the number of concepts to be taught and possible teacher strategies are small, they perform poorly in less constrained scenarios. Our human quality evaluation shows that both models and ground-truth annotations exhibit low performance in terms of equitable tutoring, which measures learning opportunities for students and how engaging the dialog is. To understand the behavior of our models in a real tutoring setting, we conduct a user study using expert annotators and find a significantly large number of model reasoning errors in 45% of conversations. Finally, we connect our findings to outline future work.