# A randomization-based perspective of analysis of variance: a test statistic robust to treatment effect heterogeneity

arXiv: https://arxiv.org/abs/1608.01787 · HTML: https://mlsanity.com/paper/1608.01787/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Aug 2016.

## Built on by (most cited first, 1 of 1)

- [Design Stability in Adaptive Experiments: Implications for Treatment Effect Estimation](https://mlsanity.com/paper/2510.22351/) (2025)

## Abstract (arXiv)

Fisher randomization tests for Neyman's null hypothesis of no average treatment effects are considered in a finite population setting associated with completely randomized experiments with more than two treatments. The consequences of using the F statistic to conduct such a test are examined both theoretically and computationally, and it is argued that under treatment effect heterogeneity, use of the F statistic in the Fisher randomization test can severely inflate the type I error under Neyman's null hypothesis. An alternative test statistic is proposed, its asymptotic distributions under Fisher's and Neyman's null hypotheses are derived, and its advantages demonstrated.