# Cutting Through the Noise: Boosting LLM Performance on Math Word Problems

arXiv: https://arxiv.org/abs/2406.15444 · HTML: https://mlsanity.com/paper/2406.15444/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 30 May 2024.

## Builds on (most cited first, 26 of 26)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [Yi: Open Foundation Models by 01.AI](https://mlsanity.com/paper/2403.04652/) (2024)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [Solving General Arithmetic Word Problems](https://mlsanity.com/paper/1608.01413/) (2016)
- [MathPrompter: Mathematical Reasoning using Large Language Models](https://mlsanity.com/paper/2303.05398/) (2023)
- [An LLM can Fool Itself: A Prompt-Based Adversarial Attack](https://mlsanity.com/paper/2310.13345/) (2023)
- [ARB: Advanced Reasoning Benchmark for Large Language Models](https://mlsanity.com/paper/2307.13692/) (2023)
- [Mapping to Declarative Knowledge for Word Problem Solving](https://mlsanity.com/paper/1712.09391/) (2017)
- [MathAttack: Attacking Large Language Models Towards Math Solving Ability](https://mlsanity.com/paper/2309.01686/) (2023)
- [An Independent Evaluation of ChatGPT on Mathematical Word Problems (MWP)](https://mlsanity.com/paper/2302.13814/) (2023)
- [Adversarial Examples for Evaluating Math Word Problem Solvers](https://mlsanity.com/paper/2109.05925/) (2021)
- [Reasoning in Large Language Models Through Symbolic Math Word Problems](https://mlsanity.com/paper/2308.01906/) (2023)
- ["John is 50 years old, can his son be 65?" Evaluating NLP Models' Understanding of Feasibility](https://mlsanity.com/paper/2210.07471/) (2022)
- [Multi stain graph fusion for multimodal integration in pathology](https://mlsanity.com/paper/2204.12541/) (2022)
- [Practice Makes a Solver Perfect: Data Augmentation for Math Word Problem Solvers](https://mlsanity.com/paper/2205.00177/) (2022)
- [MAEA: Multimodal Attribution for Embodied AI](https://mlsanity.com/paper/2307.13850/) (2023)
- [Deep Kernels with Probabilistic Embeddings for Small-Data Learning](https://mlsanity.com/paper/1910.05858/) (2019)
- [Semantic Augmentation in Images using Language](https://mlsanity.com/paper/2404.02353/) (2024)
- [Attribution Regularization for Multimodal Paradigms](https://mlsanity.com/paper/2404.02359/) (2024)
- [Intent Detection for code-mix utterances in task oriented dialogue systems](https://mlsanity.com/paper/1812.02914/) (2018)
- [Retraining DistilBERT for a Voice Shopping Assistant by Using Universal Dependencies](https://mlsanity.com/paper/2103.15737/) (2021)

## Built on by (most cited first, 14 of 14)

- [Not All LLM Reasoners Are Created Equal](https://mlsanity.com/paper/2410.01748/) (2024)
- [How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark](https://mlsanity.com/paper/2505.18761/) (2025)
- [Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Models](https://mlsanity.com/paper/2503.01781/) (2025)
- [UTMath: Math Evaluation with Unit Test via Reasoning-to-Coding Thoughts](https://mlsanity.com/paper/2411.07240/) (2024)
- [Are Language Models Efficient Reasoners? A Perspective from Logic Programming](https://mlsanity.com/paper/2510.25626/) (2025)
- [Improving Math Problem Solving in Large Language Models Through Categorization and Strategy Tailoring](https://mlsanity.com/paper/2411.00042/) (2024)
- [Domain Specific Benchmarks for Evaluating Multimodal Large Language Models](https://mlsanity.com/paper/2506.12958/) (2025)
- [Language Model as Planner and Formalizer under Constraints](https://mlsanity.com/paper/2510.05486/) (2025)
- [Robust Reasoning Benchmark](https://mlsanity.com/paper/2604.08571/) (2026)
- [Iterative LLM-Based Generation and Refinement of Distracting Conditions in Math Word Problems](https://mlsanity.com/paper/2510.08615/) (2025)
- [Large Language Models for Math Education in Low-Resource Languages: A Study in Sinhala and Tamil](https://mlsanity.com/paper/2602.14517/) (2026)
- [Disentangling Mathematical Reasoning in LLMs: A Methodological Investigation of Internal Mechanisms](https://mlsanity.com/paper/2604.15842/) (2026)
- [When Correct Demonstrations Hurt: Rethinking the Role of Exemplars in In-Context Learning](https://mlsanity.com/paper/2605.26350/) (2026)
- [DenoiseRL: Bootstrapping Reasoning Models to Recover from Noisy Prefixes](https://mlsanity.com/paper/2605.28421/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) excel at various tasks, including solving math word problems (MWPs), but struggle with real-world problems containing irrelevant information. To address this, we propose a prompting framework that generates adversarial variants of MWPs by adding irrelevant variables. We introduce a dataset, PROBLEMATHIC, containing both adversarial and non-adversarial MWPs. Our experiments reveal that LLMs are susceptible to distraction by numerical noise, resulting in an average relative performance drop of ~26% on adversarial MWPs. To mitigate this, we fine-tune LLMs (Llama-2, Mistral) on the adversarial samples from our dataset. Fine-tuning on adversarial training instances improves performance on adversarial MWPs by ~8%, indicating increased robustness to noise and improved ability to identify relevant data for reasoning. Finally, to assess the generalizability of our prompting framework, we introduce GSM-8K-Adv, an adversarial variant of the GSM-8K benchmark. LLMs continue to struggle when faced with adversarial information, reducing performance by up to 6%.