# Near-Optimal Non-Parametric Sequential Tests and Confidence Sequences with Possibly Dependent Observations

arXiv: https://arxiv.org/abs/2212.14411 · HTML: https://mlsanity.com/paper/2212.14411/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 80 references that are in the map.
- First posted on arXiv 29 Dec 2022.

## Builds on (most cited first, 13 of 13)

- [Time-uniform, nonparametric, nonasymptotic confidence sequences](https://mlsanity.com/paper/1810.08240/) (2018)
- [Game-theoretic statistics and safe anytime-valid inference](https://mlsanity.com/paper/2210.01948/) (2022)
- [Safe Testing](https://mlsanity.com/paper/1906.07801/) (2019)
- [Admissible anytime-valid sequential inference must rely on nonnegative martingales](https://mlsanity.com/paper/2009.03167/) (2020)
- [Universal Inference](https://mlsanity.com/paper/1912.11436/) (2019)
- [Testing exchangeability: fork-convexity, supermartingales, and e-processes](https://mlsanity.com/paper/2102.00630/) (2021)
- [Post-Contextual-Bandit Inference](https://mlsanity.com/paper/2106.00418/) (2021)
- [Sequential estimation of quantiles with applications to A/B-testing and best-arm identification](https://mlsanity.com/paper/1906.09712/) (2019)
- [Risk Minimization from Adaptively Collected Data: Guarantees for Supervised and Policy Learning](https://mlsanity.com/paper/2106.01723/) (2021)
- [A composite generalization of Ville's martingale theorem using e-processes](https://mlsanity.com/paper/2203.04485/) (2022)
- [Rapid Regression Detection in Software Deployments through Sequential Testing](https://mlsanity.com/paper/2205.14762/) (2022)
- [Anytime-Valid Inference for Multinomial Count Data](https://mlsanity.com/paper/2011.03567/) (2020)
- [Optional Stopping with Bayes Factors: a categorization and extension of folklore results, with an application to invariant situations](https://mlsanity.com/paper/1807.09077/) (2018)

## Built on by (most cited first, 9 of 9)

- [Peeking with PEAK: Sequential, Nonparametric Composite Hypothesis Tests for Means of Multiple Data Streams](https://mlsanity.com/paper/2402.06122/) (2024)
- [Anytime-valid, Bayes-assisted, Prediction-Powered Inference](https://mlsanity.com/paper/2505.18000/) (2025)
- [Simulation-Based Inference for Adaptive Experiments](https://mlsanity.com/paper/2506.02881/) (2025)
- [Stop Guessing When to Stop Testing: Efficient Model Evaluation with Just Enough Data](https://mlsanity.com/paper/2607.08522/) (2026)
- [A new and flexible class of sharp asymptotic time-uniform confidence sequences](https://mlsanity.com/paper/2502.10380/) (2025)
- [Efficient Inference after Directionally Stable Adaptive Experiments](https://mlsanity.com/paper/2602.21478/) (2026)
- [Online Learning of Optimal Sequential Testing Policies](https://mlsanity.com/paper/2509.03707/) (2025)
- [GAAVI: Global Asymptotic Anytime Valid Inference for the Conditional Mean Function](https://mlsanity.com/paper/2602.08096/) (2026)
- [Confidence Horizons](https://mlsanity.com/paper/2608.03889/) (2026)

## Abstract (arXiv)

Sequential tests and their implied confidence sequences, which are valid at arbitrary stopping times, promise flexible statistical inference and on-the-fly decision making. However, strong guarantees are limited to parametric sequential tests that under-cover in practice or concentration-bound-based sequences that over-cover and have suboptimal rejection times. In this work, we consider classic delayed-start normal-mixture sequential probability ratio tests, and we provide the first asymptotic type-I-error and expected-rejection-time guarantees under general non-parametric data generating processes, where the asymptotics are indexed by the test's burn-in time. The type-I-error results primarily leverage a martingale strong invariance principle and establish that these tests (and their implied confidence sequences) have type-I error rates asymptotically equivalent to the desired (possibly varying) α-level. The expected-rejection-time results primarily leverage an identity inspired by Ito's lemma and imply that, in certain asymptotic regimes, the expected rejection time is asymptotically equivalent to the minimum possible among α-level tests. We show how to apply our results to sequential inference on parameters defined by estimating equations, such as average treatment effects. Together, our results establish these (ostensibly parametric) tests as general-purpose, non-parametric, and near-optimal. We illustrate this via numerical simulations and a real-data application to A/B testing at Netflix.