mlsanityData to 14 Oct 2026

What Makes Math Word Problems Challenging for LLMs?

arXiv 2403.11369 ↗First posted 17 Mar 2024cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

This paper investigates the question of what makes math word problems (MWPs) in English challenging for large language models (LLMs). We conduct an in-depth analysis of the key linguistic and mathematical characteristics of MWPs. In addition, we train feature-based classifiers to better understand the impact of each feature on the overall difficulty of MWPs for prominent LLMs and investigate whether this helps predict how well LLMs fare against specific categories of MWPs.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 8 of its 22 references that are in the map. First posted on arXiv 17 Mar 2024.

Builds on

8 of 22 references.

References · 8

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  6. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  7. 2023Learning Multi-Step Reasoning by Solving Arithmetic Taskscited by 12
  8. 2023Math Word Problem Solving by Generating Linguistic Variants of Problem Statementscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 10; 2026: 3.

19 papers.

Sort:

By area:

  1. 2024Harnessing the Power of Multiple Minds: Lessons Learned from LLM Routingcited by 46
  2. 2025MathFusion: Enhancing Mathematical Problem-solving of LLM through Instruction Fusioncited by 30
  3. 2025Reasoning and Sampling-Augmented MCQ Difficulty Prediction via LLMscited by 14
  4. 2024Exposing the Achilles' Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoningcited by 13
  5. 2024SelectLLM: Query-Aware Efficient Selection Algorithm for Large Language Modelscited by 8
  6. 2025LLMs cannot spot math errors, even when allowed to peek into the solutioncited by 7
  7. 2025Knowledge Augmented Complex Problem Solving with Large Language Models: A Surveycited by 6
  8. 2024Learning by Analogy: Enhancing Few-Shot Prompting for Math Word Problem Solving with Computational Graph-Based Retrievalcited by 5
  9. 2024Cracking the Code: Multi-domain LLM Evaluation on Real-World Professional Exams in Indonesiacited by 4
  10. 2025What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patternscited by 4
  11. 2024Improving Math Problem Solving in Large Language Models Through Categorization and Strategy Tailoringcited by 3
  12. 2025Simulating Student-Tutor Interactions with LLMs for Multilingual Math Feedbackcited by 3
7 more
  1. 2026GSM-SEM: Benchmark and Framework for Generating Semantically Variant Augmentationscited by 2
  2. 2025HalluCounter: Reference-free LLM Hallucination Detection in the Wild!cited by 1
  3. 2025Exploring Task Performance with Interpretable Models via Sparse Auto-Encoderscited by 1
  4. 2025Can LLMs Solve ASP Problems? Insights from a Benchmarking Study (Extended Version)cited by 1
  5. 2025Bridging the Culture Gap: A Framework for LLM-Driven Socio-Cultural Localization of Math Word Problems in Low-Resource Languagescited by 1
  6. 2026Large Language Models for Math Education in Low-Resource Languages: A Study in Sinhala and Tamilcited by 0
  7. 2026When Tokenization is Secretly Output Supervisioncited by 0