# Anytime-Valid Inference for Multinomial Count Data

arXiv: https://arxiv.org/abs/2011.03567 · HTML: https://mlsanity.com/paper/2011.03567/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2020.

## Built on by (most cited first, 5 of 5)

- [Near-Optimal Non-Parametric Sequential Tests and Confidence Sequences with Possibly Dependent Observations](https://mlsanity.com/paper/2212.14411/) (2022)
- [An Experimental Design for Anytime-Valid Causal Inference on Multi-Armed Bandits](https://mlsanity.com/paper/2311.05794/) (2023)
- [Anytime-Valid Inference for Double/Debiased Machine Learning of Causal Parameters](https://mlsanity.com/paper/2408.09598/) (2024)
- [Best-of-∞ -- Asymptotic Performance of Test-Time LLM Ensembling](https://mlsanity.com/paper/2509.21091/) (2025)
- [Design-Based Bandits Under Network Interference: Trade-Off Between Regret and Statistical Inference](https://mlsanity.com/paper/2510.07646/) (2025)

## Abstract (arXiv)

Many experiments are concerned with the comparison of counts between treatment groups. Examples include the number of successful signups in conversion rate experiments, or the number of errors produced by software versions in canary experiments. Observations typically arrive in data streams and practitioners wish to continuously monitor their experiments, sequentially testing hypotheses while maintaining Type I error probabilities under optional stopping and continuation. These goals are frequently complicated in practice by non-stationary time dynamics. We provide practical solutions through sequential tests of multinomial hypotheses, hypotheses about many inhomogeneous Bernoulli processes and hypotheses about many time-inhomogeneous Poisson counting processes. For estimation, we further provide confidence sequences for multinomial probability vectors, all contrasts among probabilities of inhomogeneous Bernoulli processes and all contrasts among intensities of time-inhomogeneous Poisson counting processes. Together, these provide an "anytime-valid" inference framework for a wide variety of experiments dealing with count outcomes, which we illustrate with a number of industry applications.