mlsanityData to 7 Oct 2026

Safe Testing

arXiv 1906.07801 ↗First posted 18 Jun 2019math.ST cs.IT cs.LGcited by 98

Not on the map. Why?

Abstract (arXiv)

We develop the theory of hypothesis testing based on the e-value, a notion of evidence that, unlike the p-value, allows for effortlessly combining results from several studies in the common scenario where the decision to perform a new study may depend on previous outcomes. Tests based on e-values are safe, i.e. they preserve Type-I error guarantees, under such optional continuation. We define growth-rate optimality (GRO) as an analogue of power in an optional continuation context, and we show how to construct GRO e-variables for general testing problems with composite null and alternative, emphasizing models with nuisance parameters. GRO e-values take the form of Bayes factors with special priors. We illustrate the theory using several classic examples including a one-sample safe t-test and the 2 x 2 contingency table. Sharing Fisherian, Neymanian and Jeffreys-Bayesian interpretations, e-values may provide a methodology acceptable to adherents of all three schools.

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 9; 2023: 12; 2024: 11; 2025: 18; 2026: 43.

98 papers.

Sort:

By area:

  1. 2021Tight Concentrations and Confidence Sequences from the Regret of Universal Portfoliocited by 42
  2. 2022Anytime-valid off-policy inference for contextual banditscited by 37
  3. 2022E-detectors: a nonparametric framework for sequential change detectioncited by 27
  4. 2023A unified recipe for deriving (time-uniform) PAC-Bayes boundscited by 25
  5. 2023Auditing Fairness by Bettingcited by 24
  6. 2024Protected Test-Time Adaptation via Online Entropy Matching: A Betting Approachcited by 22
  7. 2024Enhancing Conformal Prediction Using E-Test Statisticscited by 16
  8. 2022Sequential Kernelized Independence Testingcited by 15
  9. 2022Model-X Sequential Testing for Conditional Independence via Testing by Bettingcited by 14
  10. 2024Active, anytime-valid risk controlling prediction setscited by 14
  11. 2025E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testingcited by 14
  12. 2022Principal-Agent Hypothesis Testingcited by 13
38 more
  1. 2025Backward Conformal Predictioncited by 13
  2. 2023Online multiple testing with e-valuescited by 12
  3. 2022E-Valuating Classifier Two-Sample Testscited by 10
  4. 2024Peeking with PEAK: Sequential, Nonparametric Composite Hypothesis Tests for Means of Multiple Data Streamscited by 9
  5. 2025On Continuous Monitoring of Risk Violations under Unknown Shiftcited by 9
  6. 2021A unified framework for bandit multiple testingcited by 8
  7. 2021Comparing Sequential Forecasterscited by 8
  8. 2022Multi-disciplinary fairness considerations in machine learning for clinical trialscited by 8
  9. 2022Near-Optimal Non-Parametric Sequential Tests and Confidence Sequences with Possibly Dependent Observationscited by 8
  10. 2023Huber-Robust Confidence Sequencescited by 7
  11. 2023Deep anytime-valid hypothesis testingcited by 7
  12. 2022Nonparametric extensions of randomized response for private confidence setscited by 6
  13. 2023Risk-limiting Financial Audits via Weighted Sampling without Replacementcited by 6
  14. 2026Resolution Diagnostics for Paired LLM Evaluationcited by 6
  15. 2025Post-detection inference for sequential changepoint localizationcited by 5
  16. 2021Conformal testing in a binary model situationcited by 4
  17. 2025Optimistic Interior Point Methods for Sequential Hypothesis Testing by Bettingcited by 4
  18. 2025Adaptive Coverage Policies in Conformal Predictioncited by 4
  19. 2025Root Cause Analysis of Outliers in Unknown Cyclic Graphscited by 4
  20. 2026When Can Conformal Risk Control Certify LLM Outputs? Bounds, Impossibility, and Adaptation for Structured Generationcited by 4
  21. 2023Designing monitoring strategies for deployed machine learning algorithms: navigating performativity through a causal lenscited by 3
  22. 2024Asymptotic Time-Uniform Inference for Parameters in Averaged Stochastic Approximationcited by 3
  23. 2025Toward Universal Laws of Outlier Propagationcited by 3
  24. 2025Conformalized Decision Risk Assessmentcited by 3
  25. 2025Anytime-Valid Answer Sufficiency Certificates for LLM Generation via Sequential Information Liftcited by 3
  26. 2026Efficient Sequential Evaluation of Large Language Modelscited by 3
  27. 2021Generic E-Variables for Exact Sequential k-Sample Tests that allow for Optional Stoppingcited by 2
  28. 2023Optimal tests following sequential experimentscited by 2
  29. 2023Counterfactually Comparing Abstaining Classifierscited by 2
  30. 2023Anytime-valid t-tests and confidence sequences for Gaussian means with unknown variancecited by 2
  31. 2024Combining Evidence Across Filtrationscited by 2
  32. 2024Online-to-PAC generalization bounds under graph-mixing dependenciescited by 2
  33. 2025Fuzzy Prediction Sets: Conformal Prediction with E-valuescited by 2
  34. 2026Asymptotically optimal sequential change detection for bounded meanscited by 2
  35. 2026Adaptive auditing of AI systems with anytime-valid guaranteescited by 2
  36. 2026Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretabilitycited by 2
  37. 2023Valid Inference for Machine Learning Model Parameterscited by 1
  38. 2024Early Stopping Based on Repeated Significancecited by 1

50 most cited of 98 All 98 (JSON)