# Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges

arXiv: https://arxiv.org/abs/2502.08680 · HTML: https://mlsanity.com/paper/2502.08680/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Mathematical reasoning, one of 17 directions in LLM reasoning.
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Placed in

- LLM reasoning / [Mathematical reasoning](https://mlsanity.com/lr/mathematical-reasoning/)

## Builds on (most cited first, 26 of 26)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models](https://mlsanity.com/paper/2410.05229/) (2024)
- [Scaling Laws for Autoregressive Generative Modeling](https://mlsanity.com/paper/2010.14701/) (2020)
- [Do LLMs exhibit human-like response biases? A case study in survey design](https://mlsanity.com/paper/2311.04076/) (2023)
- [How well do Large Language Models perform in Arithmetic tasks?](https://mlsanity.com/paper/2304.02015/) (2023)
- [Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango](https://mlsanity.com/paper/2209.07686/) (2022)
- [Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance](https://mlsanity.com/paper/2305.17306/) (2023)
- [A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models](https://mlsanity.com/paper/2210.12023/) (2022)
- [GPT Can Solve Mathematical Problems Without a Calculator](https://mlsanity.com/paper/2309.03241/) (2023)
- [MR-GSM8K: A Meta-Reasoning Benchmark for Large Language Model Evaluation](https://mlsanity.com/paper/2312.17080/) (2023)
- [Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correction](https://mlsanity.com/paper/2406.00755/) (2024)
- [An Independent Evaluation of ChatGPT on Mathematical Word Problems (MWP)](https://mlsanity.com/paper/2302.13814/) (2023)
- [Investigating Implicit Bias in Large Language Models: A Large-Scale Study of Over 50 LLMs](https://mlsanity.com/paper/2410.12864/) (2024)
- [How Numerical Precision Affects Arithmetical Reasoning Capabilities of LLMs](https://mlsanity.com/paper/2410.13857/) (2024)
- [Evaluating LLMs' Mathematical and Coding Competency through Ontology-guided Interventions](https://mlsanity.com/paper/2401.09395/) (2024)
- [Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicates](https://mlsanity.com/paper/2402.05980/) (2024)
- [ArithmAttack: Evaluating Robustness of LLMs to Noisy Context in Math Problem Solving](https://mlsanity.com/paper/2501.08203/) (2025)
- [Mathify: Evaluating Large Language Models on Mathematical Problem Solving Tasks](https://mlsanity.com/paper/2404.13099/) (2024)
- [Learning Beyond Pattern Matching? Assaying Mathematical Understanding in LLMs](https://mlsanity.com/paper/2405.15485/) (2024)
- [Solving for X and Beyond: Can Large Language Models Solve Complex Math Problems with More-Than-Two Unknowns?](https://mlsanity.com/paper/2407.05134/) (2024)
- [Adversarial Math Word Problem Generation](https://mlsanity.com/paper/2402.17916/) (2024)

## Built on by (most cited first, 24 of 24)

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [From System 1 to System 2: A Survey of Reasoning Large Language Models](https://mlsanity.com/paper/2502.17419/) (2025)
- [Free-MAD: Consensus-Free Multi-Agent Debate](https://mlsanity.com/paper/2509.11035/) (2025)
- [FoNE: Precise Single-Token Number Embeddings via Fourier Features](https://mlsanity.com/paper/2502.09741/) (2025)
- [Pi-GPS: Enhancing Geometry Problem Solving by Unleashing the Power of Diagrammatic Information](https://mlsanity.com/paper/2503.05543/) (2025)
- [MAD-Spear: A Conformity-Driven Prompt Injection Attack on Multi-Agent Debate Systems](https://mlsanity.com/paper/2507.13038/) (2025)
- [Process or Result? Manipulated Ending Tokens Can Mislead Reasoning LLMs to Ignore the Correct Reasoning Steps](https://mlsanity.com/paper/2503.19326/) (2025)
- [Benford's Curse: Tracing Digit Bias to Numerical Hallucination in LLMs](https://mlsanity.com/paper/2506.01734/) (2025)
- [ASyMOB: Algebraic Symbolic Mathematical Operations Benchmark](https://mlsanity.com/paper/2505.23851/) (2025)
- [Token-Efficient Prompt Injection Attack: Provoking Cessation in LLM Reasoning via Adaptive Token Compression](https://mlsanity.com/paper/2504.20493/) (2025)
- [DataPuzzle: Breaking Free from the Hallucinated Promise of LLMs in Data Analysis](https://mlsanity.com/paper/2504.10036/) (2025)
- [Numerical Sensitivity and Robustness: Exploring the Flaws of Mathematical Reasoning in Large Language Models](https://mlsanity.com/paper/2511.08022/) (2025)
- [Towards Probabilistic Question Answering Over Tabular Data](https://mlsanity.com/paper/2506.20747/) (2025)
- [RIDE: Difficulty Evolving Perturbation with Item Response Theory for Mathematical Reasoning](https://mlsanity.com/paper/2511.04120/) (2025)
- [MSCR: Exploring the Vulnerability of LLMs' Mathematical Reasoning Abilities Using Multi-Source Candidate Replacement](https://mlsanity.com/paper/2511.08055/) (2025)
- [A model of errors in transformers](https://mlsanity.com/paper/2601.14175/) (2026)
- [Agent Harness Distillation: Inference-Time Harness Extraction and Exploitation in Autonomous Multi-Agent Systems](https://mlsanity.com/paper/2607.28147/) (2026)
- [Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs](https://mlsanity.com/paper/2506.17630/) (2025)
- [VeRA: Renewing Reasoning Benchmarks with Executable Specifications](https://mlsanity.com/paper/2602.13217/) (2026)
- [The Importance of Being Statistically Earnest: A Critical Re-evaluation of GSM-Symbolic](https://mlsanity.com/paper/2605.28700/) (2026)
- [CombEval: A Framework for Evaluating Combinatorial Counting in Large Language Models](https://mlsanity.com/paper/2606.19788/) (2026)
- [CIFQA: A Deterministic Tool-Grounded Multi-Agent LLM Framework for Financial Query Answering](https://mlsanity.com/paper/2608.26114/) (2026)
- [HealthLoopQA: A Context-Aware Question Answering Benchmark for Interpreting Wearable Monitoring Data in Diabetes Care](https://mlsanity.com/paper/2609.06976/) (2026)
- [MemoReason: Evaluating the Effect of Parametric Memory on Contextual Reasoning in LLMs](https://mlsanity.com/paper/2609.35312/) (2026)

## Abstract (arXiv)

Mathematical reasoning in Large Language Models (LLMs) is often evaluated using benchmarks with limited numerical ranges, failing to reflect real-world problem-solving across diverse scales. Furthermore, most existing evaluation methods only compare model outputs to ground-truth answers, obscuring insights into reasoning processes. To address these limitations, we introduce GSM-Ranges, a dataset generator derived from GSM8K that systematically perturbs numerical values in math problems to assess model robustness across varying numerical scales. Additionally, we propose a novel grading methodology that distinguishes between logical and non-logical errors, offering a more precise evaluation of reasoning processes beyond computational accuracy. Our experiments with various models reveal a significant increase in logical error rates-up to 14 percentage points-as numerical complexity rises, demonstrating a general weakness in reasoning with out-of-distribution numerical values. Moreover, while models demonstrate high accuracy on standalone arithmetic tasks, their performance deteriorates substantially when computations are embedded within word problems. These findings provide a comprehensive evaluation of LLMs' mathematical reasoning capabilities and inform future research directions for improving numerical generalization in language models.