mlsanityData to 7 Oct 2026

Solving Math Word Problems by Combining Language Models With Symbolic Solvers

arXiv 2304.09102 ↗First posted 16 Apr 2023cs.CL cs.AIcited by 124

Not on the map. Why?

Cited by 55 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Automatically generating high-quality step-by-step solutions to math word problems has many applications in education. Recently, combining large language models (LLMs) with external tools to perform complex reasoning and calculation has emerged as a promising direction for solving math word problems, but prior approaches such as Program-Aided Language model (PAL) are biased towards simple procedural problems and less effective for problems that require declarative reasoning. We propose an approach that combines an LLM that can incrementally formalize word problems as a set of variables and equations with an external symbolic solver that can solve the equations. Our approach achieves comparable accuracy to the original PAL on the GSM8K benchmark of math word problems and outperforms PAL by an absolute 20% on ALGEBRA, a new dataset of more challenging word problems extracted from Algebra textbooks. Our work highlights the benefits of using declarative and incremental representations when interfacing with an external tool for solving complex math word problems. Our data and prompts are publicly available at https://github.com/joyheyueya/declarative-math-word-problem.

Key facts. Cited by 124 papers in the map (data through 7 Oct 2026). Builds on 9 of its 17 references that are in the map. First posted on arXiv 16 Apr 2023.

Builds on

9 of 17 references.

References · 9

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  8. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  9. 2022PAL: Program-aided Language Modelscited by 635

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 24; 2024: 68; 2025: 24; 2026: 8.

124 papers.

Sort:

By area:

  1. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  2. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
  3. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 249
  4. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  5. 2024Tool Learning with Large Language Models: A Surveycited by 221
  6. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 140
  7. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretrainingcited by 123
  8. 2023SatLM: Satisfiability-Aided Language Models Using Declarative Promptingcited by 100
  9. 2024Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solvingcited by 82
  10. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  11. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  12. 2023Grammar Prompting for Domain-Specific Language Generation with Large Language Modelscited by 58
38 more
  1. 2024Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thoughtcited by 56
  2. 2024Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Taskscited by 52
  3. 2024Autonomous LLM-driven research from data to human-verifiable research paperscited by 50
  4. 2023Evaluating and Improving Tool-Augmented Computation-Intensive Math Reasoningcited by 49
  5. 2024LLMs for Mathematical Modeling: Towards Bridging the Gap between Natural and Mathematical Languagescited by 46
  6. 2024Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Tracescited by 40
  7. 2023OptiMUS: Optimization Modeling Using MIP Solvers and large language modelscited by 39
  8. 2024API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMscited by 39
  9. 2023MechGPT, a language-based strategy for mechanics and materials modeling that connects knowledge across scales, disciplines and modalitiescited by 38
  10. 2024Preference Optimization for Reasoning with Pseudo Feedbackcited by 38
  11. 2023Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Modelscited by 37
  12. 2024Capabilities of Large Language Models in Control Engineering: A Benchmark Study on GPT-4, Claude 3 Opus, and Gemini 1.0 Ultracited by 36
  13. 2024Planning Anything with Rigor: General-Purpose Zero-Shot Planning with LLM-based Formalized Programmingcited by 35
  14. 2024Efficient Tool Use with Chain-of-Abstraction Reasoningcited by 33
  15. 2025MCIP: Protecting MCP Safety via Model Contextual Integrity Protocolcited by 33
  16. 2024INMS: Memory Sharing for Large Language Model based Agentscited by 31
  17. 2023MeLM, a generative pretrained language modeling framework that solves forward and inverse mechanics problemscited by 29
  18. 2025Adaptive Tool Use in Large Language Models with Meta-Cognition Triggercited by 29
  19. 2024LeanReasoner: Boosting Complex Logical Reasoning with Leancited by 28
  20. 2023Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughtscited by 27
  21. 2024Octopus: On-device language model for function calling of software APIscited by 26
  22. 2024What Are the Odds? Language Models Are Capable of Probabilistic Reasoningcited by 25
  23. 2023Mathematical Language Models: A Surveycited by 24
  24. 2024Steering Large Language Models between Code Execution and Textual Reasoningcited by 24
  25. 2024How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMscited by 24
  26. 2024I Think, Therefore I am: Benchmarking Awareness of Large Language Models Using AwareBenchcited by 20
  27. 2024TimeToM: Temporal Space is the Key to Unlocking the Door of Large Language Models' Theory-of-Mindcited by 20
  28. 2024Towards the Pedagogical Steering of Large Language Models for Tutoring: A Case Study with Modeling Productive Failurecited by 20
  29. 2024Deeper Insights Without Updates: The Power of In-Context Learning Over Fine-Tuningcited by 20
  30. 2024The Fusion of Large Language Models and Formal Methods for Trustworthy AI Agents: A Roadmapcited by 20
  31. 2023Unveiling Theory of Mind in Large Language Models: A Parallel to Single Neurons in the Human Braincited by 18
  32. 2024Causal Evaluation of Language Modelscited by 18
  33. 2025EduPlanner: LLM-Based Multi-Agent Systems for Customized and Intelligent Instructional Designcited by 18
  34. 2023Code Prompting: a Neural Symbolic Method for Complex Reasoning in Large Language Modelscited by 16
  35. 2024CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoningcited by 16
  36. 2024A Systematic Assessment of OpenAI o1-Preview for Higher Order Thinking in Educationcited by 16
  37. 2025ToolRM: Outcome Reward Models for Tool-Calling Large Language Modelscited by 16
  38. 2024Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agentscited by 15

50 most cited of 124 All 124 (JSON)