mlsanityData to 14 Oct 2026

I-RAVEN-X: Benchmarking Generalization and Robustness of Analogical and Mathematical Reasoning in Large Language and Reasoning Models

arXiv 2510.17496 ↗First posted 20 Oct 2025cs.LG cs.AIcited by 4

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

We introduce I-RAVEN-X, a symbolic benchmark designed to evaluate generalization and robustness in analogical and mathematical reasoning for Large Language Models (LLMs) and Large Reasoning Models (LRMs). I-RAVEN-X extends I-RAVEN by increasing operand complexity, attribute range, and introducing perceptual uncertainty. Compared to LLMs, empirical results show that LRMs achieve improved productivity and systematicity on longer reasoning relations and wider attribute ranges, respectively. However, LRMs are still significantly challenged by reasoning under uncertainty and cannot effectively explore multiple probabilistic outcomes.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 29 references that are in the map. First posted on arXiv 20 Oct 2025.

Builds on

23 of 29 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Towards Learning to Reason: Comparing LLMs with Neuro-Symbolic on Arithmetic Relations in Abstract Reasoning7 Dec
  2. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models7 Oct
  3. 2022Solving Quantitative Reasoning Problems with Language Models29 Jun

Other references · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2022Emergent Analogical Reasoning in Large Language Modelscited by 240
  7. 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 226
  8. 2021Abstraction and Analogy-Making in Artificial Intelligencecited by 103
  9. 2020Stratified Rule-Aware Network for Abstract Visual Reasoningcited by 72
  10. 2023Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Taskscited by 55
  11. 2020Scale-Localized Abstract Reasoningcited by 45
  12. 2022Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matricescited by 37
8 more
  1. 2024MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learningcited by 30
  2. 2024What is the Visual Cognition Gap between Humans and Multimodal LLMs?cited by 30
  3. 2024How Far Are We from Intelligent Visual Deductive Reasoning?cited by 29
  4. 2022Are Deep Neural Networks SMARTer than Second Graders?cited by 24
  5. 2023In-Context Analogical Reasoning with Pre-Trained Language Modelscited by 22
  6. 2024The Curious Case of Nonverbal Abstract Reasoning with Multi-Modal Large Language Modelscited by 17
  7. 2024Bongard in Wonderland: Visual Puzzles that Still Make AI Go Mad?cited by 16
  8. 2025Can Large Reasoning Models do Analogical Reasoning under Perceptual Uncertainty?cited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Toward Cognitive Supersensing in Multimodal Large Language Modelcited by 7
  2. 2026Procedural Refinement by LLM-driven Algorithmic Debugging for ARC-AGI-2cited by 2
  3. 2026Evaluating Relational Reasoning in LLMs with RELcited by 2
  4. 2026VeriBound: PAC-Bayesian Generalization Bounds for Process Reward Models Trained with Formal Verification Toolscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025ReliableMath: Benchmark of Reliable Mathematical Reasoning on Large Language Models
  2. 2025Putnam-AXIOM: A Functional and Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs
  3. 2024Assessing the Emergent Symbolic Reasoning Abilities of Llama Large Language Models
  4. 2026General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks
  5. 2023Assessing and Enhancing the Robustness of Large Language Models with Task Structure Variations for Logical Reasoning
  6. 2025Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges
  7. 2024MC-NEST: Enhancing Mathematical Reasoning in Large Language Models leveraging a Monte Carlo Self-Refine Tree
  8. 2025Logical Reasoning in Large Language Models: A Survey