mlsanityData to 7 Oct 2026

MathChat: Converse to Tackle Challenging Math Problems with LLM Agents

arXiv 2306.01337 ↗First posted 2 Jun 2023cs.CL stat.MLcited by 67

On the map: LLM agents and tool use → Educational agents and math problem solving

Abstract (arXiv)

Employing Large Language Models (LLMs) to address mathematical problems is an intriguing research endeavor, considering the abundance of math problems expressed in natural language across numerous science and engineering fields. LLMs, with their generalized ability, are used as a foundation model to build AI agents for different tasks. In this paper, we study the effectiveness of utilizing LLM agents to solve math problems through conversations. We propose MathChat, a conversational problem-solving framework designed for math problems. MathChat consists of an LLM agent and a user proxy agent which is responsible for tool execution and additional guidance. This synergy facilitates a collaborative problem-solving process, where the agents engage in a dialogue to solve the problems. We perform evaluation on difficult high school competition problems from the MATH dataset. Utilizing Python, we show that MathChat can further improve previous tool-using prompting methods by 6%.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use. Builds on 30 of its 50 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

30 of 50 references.

References · 30

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  12. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
18 more
  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  2. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  3. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  4. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  5. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  6. 2022PAL: Program-aided Language Modelscited by 635
  7. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 554
  8. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  9. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  10. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
  11. 2022TALM: Tool Augmented Language Modelscited by 197
  12. 2023ART: Automatic multi-step reasoning and tool-use for large language modelscited by 193
  13. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  14. 2022PromptChainer: Chaining Large Language Model Prompts through Visual Programmingcited by 134
  15. 2021A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Levelcited by 96
  16. 2022Successive Prompting for Decomposing Complex Questionscited by 94
  17. 2022SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Modelscited by 38
  18. 2023HyperTime: Hyperparameter Optimization for Combating Temporal Distribution Shiftscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 26; 2025: 25; 2026: 7.

67 papers.

Sort:

By area:

  1. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  2. 2025Memory in the Age of AI Agentscited by 285
  3. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
  4. 2023The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4cited by 112
  5. 2024SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoningcited by 90
  6. 2025Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoningcited by 67
  7. 2024StateFlow: Enhancing LLM Task-Solving through State-Driven Workflowscited by 65
  8. 2024ToolNet: Connecting Large Language Models with Massive Tools via Tool Graphcited by 59
  9. 2025LLM Agents for Education: Advances and Applicationscited by 59
  10. 2023Improving Large Language Model Fine-tuning for Solving Math Problemscited by 48
  11. 2025MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaborationcited by 39
  12. 2025Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physicscited by 33
38 more
  1. 2024CoMM: Collaborative Multi-Agent, Multi-Reasoning-Path Prompting for Complex Problem Solvingcited by 31
  2. 2024M'elange: Cost Efficient Large Language Model Serving by Exploiting GPU Heterogeneitycited by 30
  3. 2024AgentCoord: Visually Exploring Coordination Strategy for LLM-based Multi-Agent Collaborationcited by 26
  4. 2024MathChat: Benchmarking Mathematical Reasoning and Instruction Following in Multi-Turn Interactionscited by 26
  5. 2024Bringing Generative AI to Adaptive Learning in Educationcited by 24
  6. 2024How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMscited by 24
  7. 2025DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systemscited by 23
  8. 2024Can We Rely on LLM Agents to Draft Long-Horizon Plans? Let's Take TravelPlanner as an Examplecited by 22
  9. 2024An overview of domain-specific foundation model: key technologies, applications and challengescited by 19
  10. 2025VeriPlan: Integrating Formal Verification and LLMs into End-User Planningcited by 19
  11. 2024Causal Evaluation of Language Modelscited by 18
  12. 2025GNNs as Predictors of Agentic Workflow Performancescited by 17
  13. 2025Plan Then Action:High-Level Planning Guidance Reinforcement Learning for LLM Reasoningcited by 17
  14. 2024DEBATE: Devil's Advocate-Based Assessment and Text Evaluationcited by 16
  15. 2025Empowering Scientific Workflows with Federated Agentscited by 16
  16. 2024Interactive Speculative Planning: Enhance Agent Efficiency through Co-design of System and User Interfacecited by 15
  17. 2025ExCyTIn-Bench: Evaluating LLM agents on Cyber Threat Investigationcited by 15
  18. 2024MARIO Eval: Evaluate Your Math LLM with your Math LLM--A mathematical dataset evaluation toolkitcited by 14
  19. 2024CATP-LLM: Empowering Large Language Models for Cost-Aware Tool Planningcited by 14
  20. 2024Uncertainty is Fragile: Manipulating Uncertainty in Large Language Modelscited by 13
  21. 2025ArithmAttack: Evaluating Robustness of LLMs to Noisy Context in Math Problem Solvingcited by 13
  22. 2025Evolution of AI in Education: Agentic Workflowscited by 13
  23. 2024Mathify: Evaluating Large Language Models on Mathematical Problem Solving Taskscited by 12
  24. 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 11
  25. 2023AI for Mathematics: A Cognitive Science Perspectivecited by 10
  26. 2024Integrating Chemistry Knowledge in Large Language Models via Prompt Engineeringcited by 10
  27. 2023Chain-of-Thought Reasoning is a Policy Improvement Operatorcited by 8
  28. 2025Graph-Augmented Reasoning: Evolving Step-by-Step Knowledge Graph Retrieval for LLM Reasoningcited by 8
  29. 2025Characterizing and Optimizing LLM Inference Workloads on CPU-GPU Coupled Architecturescited by 8
  30. 2025OrthoRank: Token Selection via Sink Token Orthogonality for Efficient LLM inferencecited by 8
  31. 2024Challenges Faced by Large Language Models in Solving Multi-Agent Flockingcited by 7
  32. 2024PersonaMath: Boosting Mathematical Reasoning via Persona-Driven Data Augmentationcited by 7
  33. 2024Improving Small-Scale Large Language Models Function Calling for Reasoning Taskscited by 7
  34. 2024MMFactory: A Universal Solution Search Engine for Vision-Language Taskscited by 7
  35. 2025Optimizing LLM-Based Multi-Agent System with Textual Feedback: A Case Study on Software Developmentcited by 7
  36. 2025MAPS: A Multilingual Benchmark for Agent Performance and Securitycited by 6
  37. 2023GPT vs Human for Scientific Reviews: A Dual Source Review on Applications of ChatGPT in Sciencecited by 5
  38. 2025From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualizationcited by 5

50 most cited of 67 All 67 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Expanding Search Space with Diverse Prompting Agents: An Efficient Sampling Approach for LLM Mathematical Reasoning
  2. 2025Exploring Communication Strategies for Collaborative LLM Agents in Mathematical Problem-Solving
  3. 2024Can Language Models Rival Mathematics Students? Evaluating Mathematical Reasoning through Textual Manipulation and Human Experiments
  4. 2024ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline
  5. 2024MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data
  6. 2023Assessing the Impact of Prompting Methods on ChatGPT's Mathematical Capabilities
  7. 2024Assessing the Emergent Symbolic Reasoning Abilities of Llama Large Language Models
  8. 2025Large Language Models Don't Make Sense of Word Problems. A Scoping Review from a Mathematics Education Perspective