mlsanityData to 14 Oct 2026

LongReasonArena: A Long Reasoning Benchmark for Large Language Models

arXiv 2508.19363 ↗First posted 26 Aug 2025cs.CL cs.AIcited by 3

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Existing long-context benchmarks for Large Language Models (LLMs) focus on evaluating comprehension of long inputs, while overlooking the evaluation of long reasoning abilities. To address this gap, we introduce LongReasonArena, a benchmark specifically designed to assess the long reasoning capabilities of LLMs. Our tasks require models to solve problems by executing multi-step algorithms that reflect key aspects of long reasoning, such as retrieval and backtracking. By controlling the inputs, the required reasoning length can be arbitrarily scaled, reaching up to 1 million tokens of reasoning for the most challenging tasks. Extensive evaluation results demonstrate that LongReasonArena presents a significant challenge for both open-source and proprietary LLMs. For instance, Deepseek-R1 achieves only 7.5% accuracy on our task. Further analysis also reveals that the accuracy exhibits a linear decline with respect to the logarithm of the expected number of reasoning steps. Our code and data is available at https://github.com/LongReasonArena/LongReasonArena.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 10 of its 32 references that are in the map. First posted on arXiv 26 Aug 2025.

Builds on

10 of 32 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025DocPuzzle: A Process-Aware Benchmark for Evaluating Realistic Long-Context Reasoning Capabilities25 Feb
  2. 2025LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion25 Jan
  3. 2024ALR2: A Retrieve-then-Reason Framework for Long-context Question Answering4 Oct

Other references · 7

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2024OpenAI o1 System Cardcited by 1,244
  5. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  6. 2025GSM-Infinite: How Do Your LLMs Behave over Infinitely Increasing Context Length and Reasoning Complexity?cited by 17
  7. 2024Large Language Models as Code Executors: An Exploratory Studycited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Efficient Evaluation of LLM Performance with Statistical Guaranteescited by 18
  2. 2026Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Predictioncited by 2
  3. 2026Verifiable Environments Are LEGO Bricks: Recursive Composition for Reasoning Generalizationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Data
  2. 2025EffiReason-Bench: A Unified Benchmark for Evaluating and Advancing Efficient Reasoning in Large Language Models
  3. 2025InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models
  4. 2025Benchmarking Reasoning Robustness in Large Language Models
  5. 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models
  6. 2024BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack
  7. 2024metabench -- A Sparse Benchmark of Reasoning and Knowledge in Large Language Models
  8. 2024Large Language Models Can Self-Improve in Long-context Reasoning