HypoSpace: A Diagnostic Benchmark for Set-Valued Hypothesis Generation under Underdetermination and Sublinear Coverage Bounds
Not on the map. Why?
Abstract (arXiv)
Many scientific problems are underdetermined: multiple distinct hypotheses are equally consistent with the same observations. In such settings, effective inference requires not only producing valid explanations, but also systematically exploring and covering the admissible hypothesis set. We introduce HypoSpace, a benchmark that treats large language models (LLMs) as samplers over finite hypothesis spaces and evaluates them on three metrics: Validity, Uniqueness, and Recovery. HypoSpace spans three structured domains (causal graph inference, gravity-constrained 3D voxel reconstruction, and Boolean genetic interaction modeling) with deterministic validators and exactly enumerable solution spaces, plus real-world anchored case studies. Empirically, HypoSpace reveals a capability- and scale-dependent coverage failure: models can maintain high Validity while exhibiting reduced Uniqueness and Recovery as admissible hypothesis spaces become larger or more combinatorial. We further show that the analysis on stratified decoding partially mitigates this collapse, demonstrating HypoSpace's utility as a diagnostic benchmark for set-valued inference. Code is available at: https://github.com/CTT-Pavilion/_HypoSpace.
Key facts. Cited by 5 papers in the map (data through 13 Oct 2026). Builds on 13 of its 27 references that are in the map. First posted on arXiv 17 Oct 2025.
Builds on
13 of 27 references.
References · 13
- 2024The Llama 3 Herd of Modelscited by 14,486
- 2020Measuring Massive Multitask Language Understandingcited by 7,876
- 2024GPT-4o System Cardcited by 4,536
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,714
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,799
- 2025LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Modelscited by 65
- 2024A Performance Study of LLM-Generated Code on Leetcodecited by 48
- 2025HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generationcited by 36
- 2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 28
- 2024IdeaBench: Benchmarking Large Language Models for Research Idea Generationcited by 22
- 2024CreativEval: Evaluating Creativity of LLM-Based Hardware Code Generationcited by 19
- 2024Can AI Be as Creative as Humans?cited by 17
1 more
Built on by
Citing papers per year: 2025: 1; 2026: 4.
5 papers.
- 2026Unlocking LLM Creativity in Science through Analogical Reasoningcited by 6
- 2026WeatherReasonSeg: A Benchmark for Weather-Aware Reasoning Segmentation in Visual Language Modelscited by 4
- 2025Universe of Thoughts: Enabling Creative Reasoning with Large Language Modelscited by 0
- 2026Evaluating Multiple LLM Generations with Validated Task Coveragecited by 0
- 2026From Scientific Observations to Mechanisms: Benchmarking Hypothesis Generation by AI Scientistscited by 0
- 2026From Scientific Observations to Mechanisms: Benchmarking Hypothesis Generation by AI Scientists4 Oct 2026
- 2026Evaluating Multiple LLM Generations with Validated Task Coverage25 Aug 2026
- 2026Unlocking LLM Creativity in Science through Analogical Reasoning11 May 2026
- 2026WeatherReasonSeg: A Benchmark for Weather-Aware Reasoning Segmentation in Visual Language Models18 Mar 2026
- 2025Universe of Thoughts: Enabling Creative Reasoning with Large Language Models25 Nov 2025