# Improved prediction of future user activity in online A/B testing

arXiv: https://arxiv.org/abs/2402.03231 · HTML: https://mlsanity.com/paper/2402.03231/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 30 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 10 of 10)

- [A Bayesian Model for Online Activity Sample Sizes](https://mlsanity.com/paper/2111.12157/) (2021)
- [Leveraging covariate adjustments at scale in online A/B testing](https://mlsanity.com/paper/2305.01109/) (2023)
- [Posteriors, conjugacy, and exponential families for completely random measures](https://mlsanity.com/paper/1410.6843/) (2014)
- [More for less: Predicting and maximizing genetic variant discovery via Bayesian nonparametrics](https://mlsanity.com/paper/1912.05516/) (2019)
- [Datasets for Online Controlled Experiments](https://mlsanity.com/paper/2111.10198/) (2021)
- [Transform-scaled process priors for trait allocations in Bayesian nonparametrics](https://mlsanity.com/paper/2303.17844/) (2023)
- [Experimentation Platforms Meet Reinforcement Learning: Bayesian Sequential Decision-Making for Continuous Monitoring](https://mlsanity.com/paper/2304.00420/) (2023)
- [Scaled process priors for Bayesian nonparametric estimation of the unseen genetic variation](https://mlsanity.com/paper/2106.15480/) (2021)
- [Independent finite approximations for Bayesian nonparametric inference](https://mlsanity.com/paper/2009.10780/) (2020)
- [Bayesian nonparametric strategies for power maximization in rare variants association studies](https://mlsanity.com/paper/2112.02032/) (2021)

## Abstract (arXiv)

In online randomized experiments or A/B tests, accurate predictions of participant inclusion rates are of paramount importance. These predictions not only guide experimenters in optimizing the experiment's duration but also enhance the precision of treatment effect estimates. In this paper we present a novel, straightforward, and scalable Bayesian nonparametric approach for predicting the rate at which individuals will be exposed to interventions within the realm of online A/B testing. Our approach stands out by offering dual prediction capabilities: it forecasts both the quantity of new customers expected in future time windows and, unlike available alternative methods, the number of times they will be observed. We derive closed-form expressions for the posterior distributions of the quantities needed to form predictions about future user activity, thereby bypassing the need for numerical algorithms such as Markov chain Monte Carlo. After a comprehensive exposition of our model, we test its performance on experiments on real and simulated data, where we show its superior performance with respect to existing alternatives in the literature.