mlsanityData to 13 Oct 2026

HypoSpace: A Diagnostic Benchmark for Set-Valued Hypothesis Generation under Underdetermination and Sublinear Coverage Bounds

arXiv 2510.15614 ↗First posted 17 Oct 2025cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Many scientific problems are underdetermined: multiple distinct hypotheses are equally consistent with the same observations. In such settings, effective inference requires not only producing valid explanations, but also systematically exploring and covering the admissible hypothesis set. We introduce HypoSpace, a benchmark that treats large language models (LLMs) as samplers over finite hypothesis spaces and evaluates them on three metrics: Validity, Uniqueness, and Recovery. HypoSpace spans three structured domains (causal graph inference, gravity-constrained 3D voxel reconstruction, and Boolean genetic interaction modeling) with deterministic validators and exactly enumerable solution spaces, plus real-world anchored case studies. Empirically, HypoSpace reveals a capability- and scale-dependent coverage failure: models can maintain high Validity while exhibiting reduced Uniqueness and Recovery as admissible hypothesis spaces become larger or more combinatorial. We further show that the analysis on stratified decoding partially mitigates this collapse, demonstrating HypoSpace's utility as a diagnostic benchmark for set-valued inference. Code is available at: https://github.com/CTT-Pavilion/_HypoSpace.

Key facts. Cited by 5 papers in the map (data through 13 Oct 2026). Builds on 13 of its 27 references that are in the map. First posted on arXiv 17 Oct 2025.

Builds on

13 of 27 references.

References · 13

  1. 2024The Llama 3 Herd of Modelscited by 14,486
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,876
  3. 2024GPT-4o System Cardcited by 4,536
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,714
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,799
  6. 2025LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Modelscited by 65
  7. 2024A Performance Study of LLM-Generated Code on Leetcodecited by 48
  8. 2025HypoBench: Towards Systematic and Principled Benchmarking for Hypothesis Generationcited by 36
  9. 2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 28
  10. 2024IdeaBench: Benchmarking Large Language Models for Research Idea Generationcited by 22
  11. 2024CreativEval: Evaluating Creativity of LLM-Based Hardware Code Generationcited by 19
  12. 2024Can AI Be as Creative as Humans?cited by 17
1 more
  1. 2025Gravity-Bench-v1: A Benchmark on Gravitational Physics Discovery for Agentscited by 13

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.