# Should I Stop or Should I Go: Early Stopping with Heterogeneous Populations

arXiv: https://arxiv.org/abs/2306.11839 · HTML: https://mlsanity.com/paper/2306.11839/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 47 references that are in the map.
- First posted on arXiv 20 Jun 2023.

## Builds on (most cited first, 13 of 13)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [Time-uniform, nonparametric, nonasymptotic confidence sequences](https://mlsanity.com/paper/1810.08240/) (2018)
- [A Survey on Causal Inference](https://mlsanity.com/paper/2002.02770/) (2020)
- [Estimating means of bounded random variables by betting](https://mlsanity.com/paper/2010.09686/) (2020)
- [Confidence Intervals for Policy Evaluation in Adaptive Experiments](https://mlsanity.com/paper/1911.02768/) (2019)
- [Double/Debiased/Neyman Machine Learning of Treatment Effects](https://mlsanity.com/paper/1701.08687/) (2017)
- [Admissible anytime-valid sequential inference must rely on nonnegative martingales](https://mlsanity.com/paper/2009.03167/) (2020)
- [Estimating heterogeneous treatment effects with right-censored data via causal survival forests](https://mlsanity.com/paper/2001.09887/) (2020)
- [A New Framework for Online Testing of Heterogeneous Treatment Effect](https://mlsanity.com/paper/2002.03277/) (2020)
- [Online Testing of Subgroup Treatment Effects Based on Value Difference](https://mlsanity.com/paper/2109.00712/) (2021)

## Built on by (most cited first, 1 of 1)

- [From Individual Experience to Collective Evidence: A Reporting-Based Framework for Identifying Systemic Harms](https://mlsanity.com/paper/2502.08166/) (2025)

## Abstract (arXiv)

Randomized experiments often need to be stopped prematurely due to the treatment having an unintended harmful effect. Existing methods that determine when to stop an experiment early are typically applied to the data in aggregate and do not account for treatment effect heterogeneity. In this paper, we study the early stopping of experiments for harm on heterogeneous populations. We first establish that current methods often fail to stop experiments when the treatment harms a minority group of participants. We then use causal machine learning to develop CLASH, the first broadly-applicable method for heterogeneous early stopping. We demonstrate CLASH's performance on simulated and real data and show that it yields effective early stopping for both clinical trials and A/B tests.