mlsanityData to 7 Oct 2026

SKYLENAGE Technical Report: Mathematical Reasoning and Contest-Innovation Benchmarks for Multi-Level Math Evaluation

arXiv 2510.01241 ↗First posted 24 Sep 2025cs.CLcited by 0

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Large language models (LLMs) now perform strongly on many public math suites, yet frontier separation within mathematics increasingly suffers from ceiling effects. We present two complementary benchmarks: SKYLENAGE-ReasoningMATH, a 100-item, structure-aware diagnostic set with per-item metadata on length, numeric density, and symbolic complexity; and SKYLENAGE-MATH, a 150-item contest-style suite spanning four stages from high school to doctoral under a seven-subject taxonomy. We evaluate fifteen contemporary LLM variants under a single setup and analyze subject x model and grade x model performance. On the contest suite, the strongest model reaches 44% while the runner-up reaches 37%; accuracy declines from high school to doctoral, and top systems exhibit a doctoral-to-high-school retention near 79%. On the reasoning set, the best model attains 81% overall, and hardest-slice results reveal clear robustness gaps between leaders and the mid-tier. In summary, we release SKYLENAGE-ReasoningMATH and report aggregate results for SKYLENAGE-MATH; together, SKYLENAGE provides a hard, reasoning-centered and broadly covering math benchmark with calibrated difficulty and rich metadata, serving as a reference benchmark for future evaluations of mathematical reasoning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 15 of its 21 references that are in the map. First posted on arXiv 24 Sep 2025.

Builds on

15 of 21 references.

References · 15

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  7. 2024DeepSeek-V3 Technical Reportcited by 724
  8. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  9. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  10. 2025Kimi K2: Open Agentic Intelligencecited by 523
  11. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 446
  12. 2021A Diverse Corpus for Evaluating and Developing English Math Word Problem Solverscited by 314
3 more
  1. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 258
  2. 2025Efficient Speculative Decoding for Llama at Scale: Challenges and Solutionscited by 7
  3. 2025Benchmarking GPT-5 for biomedical natural language processingcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes On
  2. 2025RealMath: A Continuous Benchmark for Evaluating Language Models on Research-Level Mathematics
  3. 2026Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs
  4. 2025AMO-Bench: Large Language Models Still Struggle in High School Math Competitions
  5. 2025MathRobust-LV: Evaluation of Large Language Models' Robustness to Linguistic Variations in Mathematical Reasoning
  6. 2025Evaluating the Reasoning Abilities of LLMs on Underrepresented Mathematics Competition Problems
  7. 2024A Careful Examination of Large Language Model Performance on Grade School Arithmetic
  8. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitions