# P-value: A Bless or A Curse for Evidence-Based Studies?

arXiv: https://arxiv.org/abs/1809.08503 · HTML: https://mlsanity.com/paper/1809.08503/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2018.

## Built on by (most cited first, 4 of 4)

- ["AI enhances our performance, I have no doubt this one will do the same": The Placebo effect is robust to negative descriptions of AI](https://mlsanity.com/paper/2309.16606/) (2023)
- [Predictive Data Calibration for Linear Correlation Significance Testing](https://mlsanity.com/paper/2208.07081/) (2022)
- [How accurate is mechanobiology? A statistical test of cell force](https://mlsanity.com/paper/2412.18406/) (2024)
- [When the AI Leaves the Tailorshop: Measuring What an LLM Advisor Leaves Behind in Complex Problem Solving](https://mlsanity.com/paper/2610.00163/) (2026)

## Abstract (arXiv)

As a convention, p-value is often computed in frequentist hypothesis testing and compared with the nominal significance level of 0.05 to determine whether or not to reject the null hypothesis. The smaller the p-value, the more significant the statistical test. We consider both one-sided and two-sided hypotheses in the composite hypothesis setting. For one-sided hypothesis tests, we establish the equivalence of p-value and the Bayesian posterior probability of the null hypothesis, which renders p-value an explicit interpretation of how strong the data support the null. For two-sided hypothesis tests of a point null, we recast the problem as a combination of two one-sided hypotheses alone the opposite directions and put forward the notion of a two-sided posterior probability, which also has an equivalent relationship with the (two-sided) p-value. Extensive simulation studies are conducted to demonstrate the Bayesian posterior probability interpretation for the p-value. Contrary to common criticisms of the use of p-value in evidence-based studies, we justify its utility and reclaim its importance from the Bayesian perspective, and recommend the continual use of p-value in hypothesis testing. After all, p-value is not all that bad.