# Structural Enforcement of Statistical Rigor in AI-Driven Discovery: A Functional Architecture

arXiv: https://arxiv.org/abs/2511.06701 · HTML: https://mlsanity.com/paper/2511.06701/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 56 references that are in the map.
- First posted on arXiv 10 Nov 2025.

## Builds on (most cited first, 19 of 19)

- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [Agent Laboratory: Using LLM Agents as Research Assistants](https://mlsanity.com/paper/2501.04227/) (2025)
- [The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search](https://mlsanity.com/paper/2504.08066/) (2025)
- [Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents](https://mlsanity.com/paper/2503.24047/) (2025)
- [Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions](https://mlsanity.com/paper/2503.08979/) (2025)
- [From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery](https://mlsanity.com/paper/2508.14111/) (2025)
- [Deep Research: A Survey of Autonomous Research Agents](https://mlsanity.com/paper/2508.12752/) (2025)
- [DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively](https://mlsanity.com/paper/2509.26603/) (2025)
- [Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agents](https://mlsanity.com/paper/2502.16069/) (2025)
- [Evaluating Sakana's AI Scientist: Bold Claims, Mixed Results, and a Promising Future?](https://mlsanity.com/paper/2502.14297/) (2025)
- [Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agents](https://mlsanity.com/paper/2502.11355/) (2025)
- [ToolUniverse: An open platform for democratizing AI scientists](https://mlsanity.com/paper/2509.23426/) (2025)
- [Online Rules for Control of False Discovery Rate and False Discovery Exceedance](https://mlsanity.com/paper/1603.09000/) (2016)
- [SafeScientist: Toward Risk-Aware Scientific Discoveries by LLM Agents](https://mlsanity.com/paper/2505.23559/) (2025)
- [Online control of the false discovery rate with decaying memory](https://mlsanity.com/paper/1710.00499/) (2017)
- [Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physics](https://mlsanity.com/paper/2510.09901/) (2025)
- [Asynchronous Online Testing of Multiple Hypotheses](https://mlsanity.com/paper/1812.05068/) (2018)
- [Scientific Hypothesis Generation and Validation: Methods, Datasets, and Future Directions](https://mlsanity.com/paper/2505.04651/) (2025)
- [Tea: A High-level Language and Runtime System for Automating Statistical Analysis](https://mlsanity.com/paper/1904.05387/) (2019)

## Built on by (most cited first, 1 of 1)

- [Let the Abyss Stare Back Adaptive Falsification for Autonomous Scientific Discovery](https://mlsanity.com/paper/2603.29045/) (2026)

## Abstract (arXiv)

AI-Scientist systems risk manufacturing spurious discoveries through uncontrolled multiple testing. We present a functional architecture that enforces statistical rigor at two levels: a Haskell embedded domain-specific language (the Research monad) that makes it impossible to test a hypothesis without updating the error budget, and a declarative scaffold that fixes the data flow and the statistical test, together with an OS-level sandbox that makes validation data physically absent from the environment in which LLM-generated code runs. We treat FDR control as a formal requirement and trace it to the implementation. We ground the design in a machine-checked Lean~4 formalization of LORD online false-discovery-rate (FDR) control: we derive its error budget and prove marginal FDR control, and full FDR control when thresholds do not adapt to earlier rejections. We then verify in SPARK/Ada that the LORD thresholds, computed in IEEE~754 arithmetic, never exceed the available wealth, given a margin condition that our configurations meet by a factor of at least eight; without a margin the property fails. To our knowledge this is the first machine-checked proof of an online FDR control theorem. In simulation, the architecture holds the false discovery rate near 1% against a 5% target, where a naive approach reaches 41%. In end-to-end case studies, a valid test avoids the false discoveries a flawed one produces, yet still finds real effects when the data allow. An adversarial evaluation confirms that, inside the sandbox, generated code cannot read the held-out data even when given its exact path.