mlsanityData to 14 Oct 2026

Can LLMs textitunderstand Math? -- Exploring the Pitfalls in Mathematical Reasoning

arXiv 2505.15623 ↗First posted 21 May 2025cs.CL cs.LGcited by 1

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Large language models (LLMs) demonstrate considerable potential in various natural language tasks but face significant challenges in mathematical reasoning, particularly in executing precise, multi-step logic. However, current evaluation frameworks judge their performance solely based on accuracy, which only accounts for the final answer. This study explores these pitfalls by employing a novel evaluation framework. We propose an evaluation metric called the MAPLE score, which holistically quantifies reasoning misalignment by integrating error rates, redundancy, and validity.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 7 of its 8 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

7 of 8 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Evaluating Mathematical Reasoning Beyond Accuracy8 Apr

Other references · 6

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  5. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  6. 2021MathBERT: A Pre-Trained Model for Mathematical Formula Understandingcited by 58

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Mathematical Computation and Reasoning Errors by Large Language Modelscited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2025Do LLMs Overthink Basic Math Reasoning? Benchmarking the Accuracy-Efficiency Tradeoff in Language Models
  2. 2024Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correction
  3. 2025Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges
  4. 2024Exposing the Achilles' Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoning
  5. 2026Improving Mathematical Reasoning Capabilities in Large Language Models via Reasoning Process Error Classification
  6. 2025Large Language Models and Mathematical Reasoning Failures
  7. 2024LLMs Are Not Intelligent Thinkers: Introducing Mathematical Topic Tree Benchmark for Comprehensive Evaluation of LLMs
  8. 2026Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges