ComBench: A Benchmark for Rigorous Proof Reasoning and Constructive Realization in Olympiad-Level Combinatorics
On the map: LLM reasoning → Mathematical reasoning
Abstract (arXiv)
Combinatorics is central to Olympiad-level mathematical problem solving, requiring deep discrete reasoning, creative constructions, and rigorous structural insight. Recent evidence suggests that even today's strongest frontier models remain uneven on Olympiad combinatorics, revealing a gap in creative mathematical reasoning. We introduce ComBench, an Olympiad-level combinatorics benchmark for evaluating and diagnosing the combinatorial reasoning capabilities of large language models. ComBench contains 100 human-annotated competition-level problems organized around two complementary settings: analysis-centric problems, which primarily require rigorous mathematical arguments, and construction-centric problems, which require explicit constructions in addition to correctness justifications. The evaluation protocol combines rubric-guided proof grading with deterministic construction verification, exposing cases where proof quality and construction validity diverge. Experiments on frontier open- and closed-source models show that ComBench is far from saturated: the strongest model reaches 65.4% overall Avg. and 75.3% overall Best@4. We further find that Rigorous Proof Reasoning and Constructive Realization are distinct capabilities: Kimi-K2.6 trails GPT-5.5 on analysis-centric proof grading but surpasses it on construction-centric Best@4, while Existence and Construction problems remain consistently hardest across representative frontier models.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 33 references that are in the map. First posted on arXiv 9 Jun 2026.
Builds on
23 of 33 references.
Closest ancestors · 10
Same direction, newest first
- 2026Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling13 May
- 2026Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification16 Mar
- 2025DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoning27 Nov
- 2025Towards Robust Mathematical Reasoning3 Nov
- 2025AMO-Bench: Large Language Models Still Struggle in High School Math Competitions30 Oct
- 2025Reliable Fine-Grained Evaluation of Natural Language Math Proofs14 Oct
- 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitions29 May
- 2025CombiBench: Benchmarking LLM Capability for Combinatorial Mathematics6 May
- 2025Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Models27 Mar
- 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI7 Nov
Other references · 13
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2025Qwen3 Technical Reportcited by 8,820
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
- 2025Kimi K2: Open Agentic Intelligencecited by 567
- 2025Humanity's Last Examcited by 491
- 2025Gold-medalist Performance in Solving Olympiad Geometry with AlphaGeometry2cited by 97
- 2025Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipelinecited by 82
- 2026Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillationcited by 60
- 2025Seed-Prover 1.5: Mastering Undergraduate-Level Theorem Proving via Learning from Experiencecited by 44
- 2026Characterizing, Evaluating, and Optimizing Complex Reasoningcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification
- 2025CombiGraph-Vis: A Curated Multimodal Olympiad Benchmark for Discrete Mathematical Reasoning
- 2025A Combinatorial Identities Benchmark for Theorem Proving via Automated Theorem Generation
- 2026TCSAlgBench: Benchmarking Automated Proving for Research-Level Theoretical Computer Science
- 2026LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches
- 2026Reasoning in a Combinatorial and Constrained World: Benchmarking LLMs on Natural-Language Combinatorial Optimization
- 2026Not All Proofs Are Equal: Evaluating LLM Proof Quality Beyond Correctness
- 2025RIMO: An Easy-to-Evaluate, Hard-to-Solve Olympiad Benchmark for Advanced Mathematical Reasoning