# Early Stopping Based on Repeated Significance

arXiv: https://arxiv.org/abs/2408.00908 · HTML: https://mlsanity.com/paper/2408.00908/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 40 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 4 of 4)

- [Safe Testing](https://mlsanity.com/paper/1906.07801/) (2019)
- [Time-uniform central limit theory and asymptotic confidence sequences](https://mlsanity.com/paper/2103.06476/) (2021)
- [Ensemble Validation: Selectivity has a Price, but Variety is Free](https://mlsanity.com/paper/1610.01234/) (2016)
- [Sharp Frequency Bounds for Sample-Based Queries](https://mlsanity.com/paper/2208.06753/) (2022)

## Built on by (most cited first, 1 of 1)

- [Steady Continuous Monitoring is (Just Barely) Impossible for Tests of Unbounded Length](https://mlsanity.com/paper/2408.02821/) (2024)

## Abstract (arXiv)

For a bucket test with a single criterion for success and a fixed number of samples or testing period, requiring a p-value less than a specified value of α for the success criterion produces statistical confidence at level 1 - α. For multiple criteria, a Bonferroni correction that partitions α among the criteria produces statistical confidence, at the cost of requiring lower p-values for each criterion. The same concept can be applied to decisions about early stopping, but that can lead to strict requirements for p-values. We show how to address that challenge by requiring criteria to be successful at multiple decision points.