# Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problems

arXiv: https://arxiv.org/abs/2404.14963 · HTML: https://mlsanity.com/paper/2404.14963/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Mathematical reasoning, one of 17 directions in LLM reasoning.
- Builds on 40 of its 51 references that are in the map.
- First posted on arXiv 23 Apr 2024.

## Placed in

- LLM reasoning / [Mathematical reasoning](https://mlsanity.com/lr/mathematical-reasoning/)

## Builds on (most cited first, 30 of 40)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://mlsanity.com/paper/2309.12284/) (2023)
- [Automatic Chain of Thought Prompting in Large Language Models](https://mlsanity.com/paper/2210.03493/) (2022)
- [PAL: Program-aided Language Models](https://mlsanity.com/paper/2211.10435/) (2022)
- [Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems](https://mlsanity.com/paper/1705.04146/) (2017)
- [WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct](https://mlsanity.com/paper/2308.09583/) (2023)

## Built on by (most cited first, 30 of 31)

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimization](https://mlsanity.com/paper/2502.04306/) (2025)
- [A Survey on Large Language Models for Mathematical Reasoning](https://mlsanity.com/paper/2506.08446/) (2025)
- [The relationship between reasoning and performance in large language models--o3 (mini) thinks harder, not longer](https://mlsanity.com/paper/2502.15631/) (2025)
- [Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executors](https://mlsanity.com/paper/2504.04785/) (2025)
- [Revisiting Overthinking in Long Chain-of-Thought from the Perspective of Self-Doubt](https://mlsanity.com/paper/2505.23480/) (2025)
- [CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning](https://mlsanity.com/paper/2410.10336/) (2024)
- [Putnam-AXIOM: A Functional and Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs](https://mlsanity.com/paper/2508.08292/) (2025)
- [Read Before You Think: Mitigating LLM Comprehension Failures with Step-by-Step Reading](https://mlsanity.com/paper/2504.09402/) (2025)
- [AI Research Preference Models](https://mlsanity.com/paper/2608.13940/) (2026)
- [Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on Large Language Models](https://mlsanity.com/paper/2406.12572/) (2024)
- [ReasoningWeekly: A General Knowledge and Verbal Reasoning Challenge for Large Language Models](https://mlsanity.com/paper/2502.01584/) (2025)
- [Narrative-of-Thought: Improving Temporal Reasoning of Large Language Models via Recounted Narratives](https://mlsanity.com/paper/2410.05558/) (2024)
- [ORCH: many analyses, one merge-a deterministic multi-agent orchestrator for discrete-choice reasoning with EMA-guided routing](https://mlsanity.com/paper/2602.01797/) (2026)
- [Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA](https://mlsanity.com/paper/2509.01468/) (2025)
- [DuaShepherd: Integrating Stepwise Correctness and Potential Rewards for Mathematical Reasoning](https://mlsanity.com/paper/2506.17533/) (2025)
- [Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model](https://mlsanity.com/paper/2407.10167/) (2024)
- [Gap-Filling Prompting Enhances Code-Assisted Mathematical Reasoning](https://mlsanity.com/paper/2411.05407/) (2024)
- [Resolving Knowledge Conflicts in Domain-specific Data Selection: A Case Study on Medical Instruction-tuning](https://mlsanity.com/paper/2505.21958/) (2025)
- [Advancing Multi-Step Mathematical Reasoning in Large Language Models through Multi-Layered Self-Reflection with Auto-Prompting](https://mlsanity.com/paper/2506.23888/) (2025)
- [(P)rior(D)yna(F)low: A Priori Dynamic Workflow Construction via Multi-Agent Collaboration](https://mlsanity.com/paper/2509.14547/) (2025)
- [GLOW: Graph-Language Co-Encoding for Agentic Workflow Performance Prediction](https://mlsanity.com/paper/2512.15751/) (2025)
- [EcoThink: A Green Adaptive Inference Framework for Sustainable and Accessible Agents](https://mlsanity.com/paper/2603.25498/) (2026)
- [Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery](https://mlsanity.com/paper/2606.08728/) (2026)
- [LLM-Empowered Resource Allocation in Wireless Communications Systems](https://mlsanity.com/paper/2408.02944/) (2024)
- [Large Language Models for Math Education in Low-Resource Languages: A Study in Sinhala and Tamil](https://mlsanity.com/paper/2602.14517/) (2026)
- [X-RAY: Mapping LLM Reasoning Capability via Formalized and Calibrated Probes](https://mlsanity.com/paper/2603.05290/) (2026)
- [Adaptive Multi-Expert Reasoning via Difficulty-Aware Routing and Uncertainty-Guided Aggregation](https://mlsanity.com/paper/2604.10335/) (2026)
- [Structural Ranking of the Cognitive Plausibility of Computational Models of Analogy and Metaphors with the Minimal Cognitive Grid](https://mlsanity.com/paper/2605.01359/) (2026)
- [Imbuing Large Language Models with Bidirectional Logic for Robust Chain Repair](https://mlsanity.com/paper/2606.05030/) (2026)

## Abstract (arXiv)

Chain-of-Thought (CoT) prompting has enhanced the performance of Large Language Models (LLMs) across various reasoning tasks. However, CoT still falls short in dealing with complex math word problems, as it usually suffers from three pitfalls: semantic misunderstanding errors, calculation errors, and step-missing errors. Prior studies involve addressing the calculation errors and step-missing errors, but neglect the semantic misunderstanding errors, which is the major factor limiting the reasoning performance of LLMs. To this end, we propose a simple-yet-effective method, namely Deeply Understanding the Problems (DUP), to improve the LLMs' math problem-solving ability by addressing semantic misunderstanding errors. The core of our method is to encourage the LLMs to deeply understand the problems and extract the key problem-solving information used for better reasoning. Extensive experiments on 10 diverse reasoning benchmarks show that our DUP method consistently outperforms the other counterparts by a large margin. More encouragingly, DUP achieves a new SOTA result on the GSM8K benchmark, with an accuracy of 97.1% under the zero-shot setting.