The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scaling
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
We introduce the Complexity Ceiling Benchmark (CCB), a controlled evaluation of how language-model reasoning decays as the number of required sequential steps grows. CCB fixes the semantic content of a task and varies only its depth N in {5,...,50} across three structurally distinct regimes: grounded spatial state-tracking, abstract symbolic pointer manipulation, and transitive relational inference. Across 6,000 trials over five frontier and open-weight LLMs we find a consistent pattern of geometric per-step decay with widely separated domain ceilings: on the first two regimes the strongest models retain pd>0.92 across N=50; on the third every model collapses by N=5, with the best model's 50%-success horizon at H0.5~4.7 steps despite pd=0.863. A trace-level metric (TFBC) shows that 14.5% of correct answers across the benchmark are reached via incorrect intermediate reasoning. Forced verbose state-tracking does not move the ceiling (McNemar p=1.000), and the mean step at which reasoning first diverges, k*, predicts within-domain accuracy better than parameter count. CCB and the geometric decay model together reduce a model's long-horizon reasoning profile to one interpretable number per task family.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 15 of its 17 references that are in the map. First posted on arXiv 28 Jun 2026.
Builds on
15 of 17 references.
Closest ancestors · 2
Same direction, newest first
- 2026Quantifying Consistency in LLM Logical Reasoning via Structural Uncertainty15 Jun
- 2026Recursive Models for Long-Horizon Reasoning2 Mar
Other references · 13
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
- 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusionscited by 214
- 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 177
- 2019CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Textcited by 162
- 2025MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiencycited by 125
- 2023ReCEval: Evaluating Reasoning Chains via Correctness and Informativenesscited by 77
- 2025The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMscited by 64
- 2026WMF-AM: Probing LLM Working Memory via Depth-Parameterized Cumulative State Trackingcited by 4
- 2026TopoBench: Benchmarking LLMs on Hard Topological Reasoningcited by 3
- 2026SokoBench: Evaluating Long-Horizon Planning and Reasoning in Large Language Modelscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025seqBench: A Tunable Benchmark to Quantify Sequential Reasoning Limits of LLMs
- 2026Empirical Evidence of Complexity-Induced Limits in Large Language Models on Finite Discrete State-Space Problems with Explicit Validity Constraints
- 2026ReEfBench: Quantifying the Reasoning Efficiency of LLMs
- 2025RELIC: Evaluating Complex Reasoning via the Recognition of Languages In-Context
- 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- 2025Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Ahead
- 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
- 2024AQA-Bench: An Interactive Benchmark for Evaluating LLMs' Sequential Reasoning Ability