AlgBench: To What Extent Do Large Reasoning Models Understand Algorithms?
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Reasoning ability has become a central focus in the advancement of Large Reasoning Models (LRMs). Although notable progress has been achieved on several reasoning benchmarks such as MATH500 and LiveCodeBench, existing benchmarks for algorithmic reasoning remain limited, failing to answer a critical question: Do LRMs truly master algorithmic reasoning? To answer this question, we propose AlgBench, an expert-curated benchmark that evaluates LRMs under an algorithm-centric paradigm. AlgBench consists of over 3,000 original problems spanning 27 algorithms, constructed by ACM algorithmic experts and organized under a comprehensive taxonomy, including Euclidean-structured, non-Euclidean-structured, non-optimized, local-optimized, global-optimized, and heuristic-optimized categories. Empirical evaluations on leading LRMs (e.g., Gemini-3-Pro, DeepSeek-v3.2-Speciale and GPT-o3) reveal substantial performance heterogeneity: while models perform well on non-optimized tasks (up to 92%), accuracy drops sharply to around 49% on globally optimized algorithms such as dynamic programming. Further analysis uncovers strategic over-shifts, wherein models prematurely abandon correct algorithmic designs due to necessary low-entropy tokens. These findings expose fundamental limitations of problem-centric reinforcement learning and highlight the necessity of an algorithm-centric training paradigm for robust algorithmic reasoning.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 23 of its 36 references that are in the map. First posted on arXiv 8 Jan 2026.
Builds on
23 of 36 references.
References · 23
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
- 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
- 2021Measuring Coding Challenge Competence With APPScited by 902
- 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
- 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
- 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 559
- 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
- 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 262
- 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 243
- 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 210
11 more
- 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 209
- 2023TACO: Topics in Algorithmic COde generation datasetcited by 160
- 2025MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbationscited by 89
- 2023KoLA: Carefully Benchmarking World Knowledge of Large Language Modelscited by 85
- 2025Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiadcited by 74
- 2025LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMscited by 70
- 2025LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?cited by 55
- 2024The CLRS-Text Algorithmic Reasoning Language Benchmarkcited by 28
- 2025ProBench: Benchmarking Large Language Models in Competitive Programmingcited by 15
- 2024GraCoRe: Benchmarking Graph Comprehension and Complex Reasoning in Large Language Modelscited by 13
- 2025UGMathBench: A Diverse and Dynamic Benchmark for Undergraduate-Level Mathematical Reasoning with Large Language Modelscited by 13
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Exposing Weaknesses of Large Reasoning Models through Graph Algorithm Problems
- 2025Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approach
- 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
- 2023ARB: Advanced Reasoning Benchmark for Large Language Models
- 2025Is Long-to-Short a Free Lunch? Investigating Inconsistency and Reasoning Efficiency in LRMs
- 2025Reasoning Models Reason Well, Until They Don't
- 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey
- 2025THINK-Bench: Evaluating Thinking Efficiency and Chain-of-Thought Quality of Large Reasoning Models