mlsanityData to 7 Oct 2026

Automatic Generation of Socratic Subquestions for Teaching Math Word Problems

arXiv 2211.12835 ↗First posted 23 Nov 2022cs.CL cs.CY cs.LGcited by 42

Not on the map. Why?

Abstract (arXiv)

Socratic questioning is an educational method that allows students to discover answers to complex problems by asking them a series of thoughtful questions. Generation of didactically sound questions is challenging, requiring understanding of the reasoning process involved in the problem. We hypothesize that such questioning strategy can not only enhance the human performance, but also assist the math word problem (MWP) solvers. In this work, we explore the ability of large language models (LMs) in generating sequential questions for guiding math word problem-solving. We propose various guided question generation schemes based on input conditioning and reinforcement learning. On both automatic and human quality evaluations, we find that LMs constrained with desirable question properties generate superior questions and improve the overall performance of a math word problem solver. We conduct a preliminary user study to examine the potential value of such question generation models in the education domain. Results suggest that the difficulty level of problems plays an important role in determining whether questioning improves or hinders human performance. We discuss the future of using such questioning strategies in education.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 34 of its 65 references that are in the map. First posted on arXiv 23 Nov 2022.

Builds on

34 of 65 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  10. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  11. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  12. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
22 more
  1. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  2. 2018CoQA: A Conversational Question Answering Challengecited by 674
  3. 2018QuAC : Question Answering in Contextcited by 379
  4. 2018Generating Wikipedia by Summarizing Long Sequencescited by 333
  5. 2017Toward Controlled Generation of Textcited by 271
  6. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
  7. 2018Style Transfer Through Back-Translationcited by 124
  8. 2020Unsupervised Question Decomposition for Question Answeringcited by 118
  9. 2018Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Informationcited by 110
  10. 2020Training Question Answering Models From Synthetic Datacited by 95
  11. 2021Generate & Rank: A Multi-task Framework for Math Word Problemscited by 82
  12. 2022A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Modelscited by 68
  13. 2019Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generationcited by 57
  14. 2021Plan-then-Generate: Controlled Data-to-Text Generation via Planningcited by 52
  15. 2020Differentiable Reasoning over a Virtual Knowledge Basecited by 39
  16. 2020Unsupervised Multi-hop Question Answering by Question Generationcited by 38
  17. 2021Data-to-text Generation with Macro Planningcited by 34
  18. 2020Inquisitive Question Generation for High Level Text Comprehensioncited by 31
  19. 2019Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worldscited by 19
  20. 2022Co-VQA : Answering by Interactive Sub Question Sequencecited by 8
  21. 2020Stronger Transformers for Neural Multi-Hop Question Generationcited by 6
  22. 2022Learning to Drop Out: An Adversarial Approach to Training Sequence VAEscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 11; 2024: 13; 2025: 12; 2026: 5.

42 papers.

Sort:

By area:

  1. 2023Challenges and Applications of Large Language Modelscited by 251
  2. 2022Distilling Reasoning Capabilities into Smaller Language Modelscited by 155
  3. 2023MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problemscited by 128
  4. 2024Calibrating Large Language Models with Sample Consistencycited by 72
  5. 2024OptiBench Meets ReSocratic: Measure and Improve LLMs for Optimization Modelingcited by 66
  6. 2025MathTutorBench: A Benchmark for Measuring Open-ended Pedagogical Capabilities of LLM Tutorscited by 47
  7. 2024Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutorscited by 39
  8. 2024Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Predictioncited by 31
  9. 2024Leveraging LLM-Respondents for Item Evaluation: a Psychometric Analysiscited by 31
  10. 2023The ART of LLM Refinement: Ask, Refine, and Trustcited by 29
  11. 2023Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generationcited by 27
  12. 2024Book2Dial: Generating Teacher-Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbotscited by 25
30 more
  1. 2023Mathematical Language Models: A Surveycited by 24
  2. 2025EducationQ: Evaluating LLMs' Teaching Capabilities Through Multi-Agent Dialogue Frameworkcited by 23
  3. 2024Enhancing Critical Thinking in Education by means of a Socratic Chatbotcited by 21
  4. 2024Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failurecited by 20
  5. 2023Learning by Analogy: Diverse Questions Generation in Math Word Problemcited by 18
  6. 2023SCREWS: A Modular Framework for Reasoning with Revisionscited by 16
  7. 2024Improving Socratic Question Generation using Data Augmentation and Preference Optimizationcited by 15
  8. 2025From Problem-Solving to Teaching Problem-Solving: Aligning LLMs with Pedagogy using Reinforcement Learningcited by 15
  9. 2023Can Language Models Employ the Socratic Method? Experiments with Code Debuggingcited by 14
  10. 2025A Survey on Mathematical Reasoning and Optimization with Large Language Modelscited by 12
  11. 2023World Models for Math Story Problemscited by 11
  12. 2025Interactive Sketchpad: A Multimodal Tutoring System for Collaborative, Visual Problem-Solvingcited by 9
  13. 2024A Systematic Review on Prompt Engineering in Large Language Models for K-12 STEM Educationcited by 8
  14. 2024SIKeD: Self-guided Iterative Knowledge Distillation for mathematical reasoningcited by 6
  15. 2025Exploring the Potential of Metacognitive Support Agents for Human-AI Co-Creationcited by 6
  16. 2025From Text to Visuals: Using LLMs to Generate Math Diagrams with Vector Graphicscited by 5
  17. 2025Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewardscited by 5
  18. 2025Generating Pedagogically Meaningful Visuals for Math Word Problems: A New Benchmark and Analysis of Text-to-Image Modelscited by 4
  19. 2023Exploring the Potential of Large Language Models in Generating Code-Tracing Questions for Introductory Programming Coursescited by 3
  20. 2024Distilling LLMs' Decomposition Abilities into Compact Language Modelscited by 3
  21. 2024Semantically Aligned Question and Code Generation for Automated Insight Generationcited by 3
  22. 2025Inverse Knowledge Search over Verifiable Reasoning: Synthesizing a Scientific Encyclopedia from a Long Chains-of-Thought Knowledge Basecited by 2
  23. 2025The Imitation Game for Educational AIcited by 1
  24. 2023Teach Me How to Improve My Argumentation Skills: A Survey on Feedback in Argumentationcited by 0
  25. 2025EMAFusion: A Self-Optimizing System for Seamless LLM Selection and Integrationcited by 0
  26. 2026A self-evolving multi-role collaborative framework with fine-grained difficulty guidance for innovative mathematical problem generationcited by 0
  27. 2026Modularizing Educational LLM-Agency for Fostering Responsible Learning Assistancecited by 0
  28. 2026Tackling the Root of Misinformation by Teaching Laypeople about Logical Fallacies via Socratic Questioning and Critical Argumentationcited by 0
  29. 2026Hey Chat, Can You Teach Me? Structuring Socratic Dialogue for Human Learning in the Wildcited by 0
  30. 2026A didactical-driven teacher assistant for a dimensional modeling coursecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Socratic Students: Teaching Language Models to Learn by Asking Questions
  2. 2024Boosting Large Language Models with Socratic Method for Conversational Mathematics Teaching
  3. 2025Self-Questioning Language Models
  4. 2024Learning by Analogy: Enhancing Few-Shot Prompting for Math Word Problem Solving with Computational Graph-Based Retrieval
  5. 2025Multi-Agent Collaborative Framework For Math Problem Generation
  6. 2026Beyond Direct Answering: Aligning Educational LLMs as Socratic Guides via Heuristic Reinforcement Learning
  7. 2024The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment with Large Language Models
  8. 2023The Art of SOCRATIC QUESTIONING: Recursive Thinking with Large Language Models