Apodex Discovery: Reality Benchmarks and Environments for Evaluating and Building Discoverative Artificial Intelligence
Not on the map. Why?
Abstract (arXiv)
Apollo did not reach the Moon merely because its engineers could solve difficult equations. It succeeded by turning a distant ambition into a mission architecture of explicit objectives, simulation, verification, and repeated correction. AI now faces a similar transition: frontier models can solve difficult tasks once the problem, tools, and success criteria are specified, yet consequential real-world challenges rarely arrive in an executable or verifiable form. We introduce Apodex Discovery, a framework for building and evaluating discoverative AI through the heavy-duty solver, a system comprising a foundation model, harness, tools, and control policies that pursues extended, stateful, verifiable investigations. It has three core components. First, a problem-scouting process surveyed 561 industries across 16 sectors, assembled 423 high-value real-world problems, and selected 20 for the initial release. Second, a common environment-task-episode abstraction provides data, tools, constraints, feedback, trajectory recording, and verification of intermediate artifacts and final submissions. Third, HDS6 evaluates Tools, Repair, Alternatives, Coherence, Evidence, and Scope independently of final-task success. In AAV capsid design, Apodex surpassed the published state of the art by 7% across viability, tropism, structure prediction, and generative design. In drug repurposing and reformulation, a task-specific biomedical environment improved the mean normalized prediction score of GPT-5.5 and GPT-5.6-sol by 2.5 and 7.6 points over the same closed-book backbone. Controlled ablations show that the fixed TRACES episode interface enables attribution of performance differences to specific solver components. Apodex Discovery moves AI evaluation beyond predefined benchmarks toward verifiable investigations aimed at genuine discovery.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 34 of its 90 references that are in the map. First posted on arXiv 11 Aug 2026.
Builds on
34 of 90 references.
References · 34
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2016OpenAI Gymcited by 1,736
- 2016Concrete Problems in AI Safetycited by 1,642
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,071
- 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
- 2022Solving math word problems with process- and outcome-based feedbackcited by 901
22 more
- 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
- 2025Humanity's Last Examcited by 491
- 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 480
- 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 469
- 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
- 2025Accelerating scientific discovery with Co-Scientistcited by 401
- 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 367
- 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
- 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 321
- 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 243
- 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 235
- 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 175
- 2024Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Modelscited by 164
- 2025GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Taskscited by 158
- 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 156
- 2024LLM Critics Help Catch LLM Bugscited by 147
- 2024DiscoveryBench: Towards Data-Driven Discovery with Large Language Modelscited by 109
- 2025SWE-Lancer: Can Frontier LLMs Earn 1 Million from Real-World Freelance Software Engineering?cited by 105
- 2026Agents' Last Examcited by 31
- 2026Position: Agentic Evolution is the Path to Evolving LLMscited by 16
- 2024AAVDiff: Experimental Validation of Enhanced Viability and Diversity in Recombinant Adeno-Associated Virus (AAV) Capsids through Diffusion Generationcited by 2
- 2026AAVGen: Precision Engineering of Adeno-associated Viral Capsids for Renal Selective Targetingcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026FrontierChallenge: Evaluating Scientific Workflow Completion25 Aug 2026