# Experiments or Outcomes? Probing Scientific Feasibility in Large Language Models

arXiv: https://arxiv.org/abs/2604.18786 · HTML: https://mlsanity.com/paper/2604.18786/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 10 references that are in the map.
- First posted on arXiv 20 Apr 2026.

## Builds on (most cited first, 6 of 6)

- [Large Language Models for Automated Open-domain Scientific Hypotheses Discovery](https://mlsanity.com/paper/2309.02726/) (2023)
- [From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery](https://mlsanity.com/paper/2505.13259/) (2025)
- [Abstention vs. Hallucination: Benchmarking LLM Source Attribution for Scientific Citations](https://mlsanity.com/paper/2405.02228/) (2024)
- [Matter-of-Fact: A Benchmark for Verifying the Feasibility of Literature-Supported Claims in Materials Science](https://mlsanity.com/paper/2506.04410/) (2025)
- [Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions](https://mlsanity.com/paper/2509.02452/) (2025)
- [NSF-SciFy: Mining the NSF Awards Database for Scientific Claims](https://mlsanity.com/paper/2503.08600/) (2025)

## Abstract (arXiv)

Scientific feasibility assessment asks whether a claim is consistent with established knowledge and whether experimental evidence could support or refute it. We frame feasibility assessment as a diagnostic reasoning task in which, given a hypothesis, a model predicts feasible or infeasible and justifies its decision. We evaluate large language models (LLMs) under controlled knowledge conditions (hypothesis-only, with experiments, with outcomes, or both) and probe robustness by progressively removing portions of the experimental and/or outcome context. Across multiple LLMs and two datasets, providing outcome evidence is generally more reliable than providing experiment descriptions. Outcomes tend to improve accuracy beyond what internal knowledge alone provides, whereas experimental text can be brittle and may degrade performance when the context is incomplete. These findings clarify when experimental evidence benefits LLM-based feasibility assessment and when it introduces fragility.