mlsanityData to 14 Oct 2026

Syntactic Blind Spots: How Misalignment Leads to LLMs Mathematical Errors

arXiv 2510.01831 ↗First posted 2 Oct 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) demonstrate strong mathematical problem-solving abilities but frequently fail on problems that deviate syntactically from their training distribution. We identify a systematic failure mode, syntactic blind spots, in which models misapply familiar reasoning strategies to problems that are semantically straightforward but phrased in unfamiliar ways. These errors are not due to gaps in mathematical competence, but rather reflect a brittle coupling between surface form and internal representation. To test this, we rephrase incorrectly answered questions using syntactic templates drawn from correct examples. These rephrasings, which preserve semantics while reducing structural complexity, often lead to correct answers. We quantify syntactic complexity using a metric based on Dependency Locality Theory (DLT), and show that higher DLT scores are associated with increased failure rates across multiple datasets. Our findings suggest that many reasoning errors stem from structural misalignment rather than conceptual difficulty, and that syntax-aware interventions can reveal and mitigate these inductive failures.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 20 of its 43 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

20 of 43 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2024Qwen2 Technical Reportcited by 2,120
  5. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  6. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  7. 2016Solving General Arithmetic Word Problemscited by 360
  8. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  9. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 318
  10. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 292
  11. 2022Data Contamination: From Memorization to Exploitationcited by 236
  12. 2024A Careful Examination of Large Language Model Performance on Grade School Arithmeticcited by 191
8 more
  1. 2024Does Prompt Formatting Have Any Impact on LLM Performance?cited by 141
  2. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 124
  3. 2024Granite Code Models: A Family of Open Foundation Models for Code Intelligencecited by 117
  4. 2025MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbationscited by 89
  5. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinatecited by 87
  6. 2024Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gapcited by 79
  7. 2023Explore Spurious Correlations at the Concept Level in Language Models for Text Classificationcited by 48
  8. 2024How Likely Do LLMs with CoT Mimic Human Reasoning?cited by 34

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026The Kernel Manifold: A Geometric Approach to Gaussian Process Model Selectioncited by 0