# OpenProblemBench: Benchmarking AI on Open Problems in the Foundational Theoretical Sciences

arXiv: https://arxiv.org/abs/2610.11118 · HTML: https://mlsanity.com/paper/2610.11118/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Builds on (most cited first, 16 of 16)

- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [PaperBench: Evaluating AI's Ability to Replicate AI Research](https://mlsanity.com/paper/2504.01848/) (2025)
- [FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI](https://mlsanity.com/paper/2411.04872/) (2024)
- [ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery](https://mlsanity.com/paper/2410.05080/) (2024)
- [SciCode: A Research Coding Benchmark Curated by Scientists](https://mlsanity.com/paper/2407.13168/) (2024)
- [AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite](https://mlsanity.com/paper/2510.21652/) (2025)
- [FrontierScience: Evaluating AI's Ability to Perform Expert-Level Scientific Tasks](https://mlsanity.com/paper/2601.21165/) (2026)
- [MLRC-Bench: Can Language Agents Solve Machine Learning Research Challenges?](https://mlsanity.com/paper/2504.09702/) (2025)
- [FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insights](https://mlsanity.com/paper/2602.02905/) (2026)
- [Benchmarking AI Agents for Addressing Scientific Challenges Across Scales](https://mlsanity.com/paper/2606.12736/) (2026)
- [UQ: Assessing Language Models on Unsolved Questions](https://mlsanity.com/paper/2508.17580/) (2025)
- [Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification](https://mlsanity.com/paper/2603.15617/) (2026)
- [How Do Agents Fail on AutoResearch: End-to-End Diagnostic Evaluation on 100 Real-World Frontier Research Tasks](https://mlsanity.com/paper/2608.14905/) (2026)
- [PRL-Bench: A Comprehensive Benchmark Evaluating LLMs' Capabilities in Frontier Physics Research](https://mlsanity.com/paper/2604.15411/) (2026)
- [TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agents](https://mlsanity.com/paper/2609.05079/) (2026)
- [Large Knowledge Model: From Papers to a Scientific Reasoning Landscape](https://mlsanity.com/paper/2609.27297/) (2026)

## Abstract (arXiv)

The next frontier for artificial general intelligence is tackling unresolved scientific problems, calling for benchmarks that assess progress beyond established knowledge. We introduce OpenProblemBench, a benchmark of 82 unresolved problems drawn from the mathematics and theoretical physics literature. Each problem supplies the research context, assumptions, and prior progress needed to investigate the question. We select problems whose proposed solutions admit comparatively clear checks of their decisive mathematical or computational claims. Four evaluator models independently assess the correctness, completeness, and degree of progress of each submission without reference solutions. Across seven evaluated configurations, GPT-6-Astra achieves the highest mean judged solve rate of 14.0%, compared with 5.5-6.7% for the evaluated full-size open models and 2.4-3.7% for Flash models. Case comparisons connect stronger outcomes to changes in problem representation, general arguments that extend beyond finite evidence, and proofs of the steps needed to complete a solution. By grounding evaluation in questions arising from the research literature, OpenProblemBench provides a setting for investigating the capabilities and limitations of AI as a contributor to foundational theoretical science.