# Adaptive multigroup confidence intervals with constant coverage

arXiv: https://arxiv.org/abs/1612.08287 · HTML: https://mlsanity.com/paper/1612.08287/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Dec 2016.

## Built on by (most cited first, 5 of 5)

- [Model-based metrics: Sample-efficient estimates of predictive model subpopulation performance](https://mlsanity.com/paper/2104.12231/) (2021)
- [FAB-PPI: Frequentist, Assisted by Bayes, Prediction-Powered Inference](https://mlsanity.com/paper/2502.02363/) (2025)
- [Trustworthy scientific inference with generative models](https://mlsanity.com/paper/2508.02602/) (2025)
- [Nonparametric Empirical Bayes Estimation and Testing for Sparse and Heteroscedastic Signals](https://mlsanity.com/paper/2106.08881/) (2021)
- [Asymptotically Log-Optimal Bayes-Assisted Confidence Sequences for Bounded Means](https://mlsanity.com/paper/2605.07964/) (2026)

## Abstract (arXiv)

Confidence intervals for the means of multiple normal populations are often based on a hierarchical normal model. While commonly used interval procedures based on such a model have the nominal coverage rate on average across a population of groups, their actual coverage rate for a given group will be above or below the nominal rate, depending on the value of the group mean. Alternatively, a coverage rate that is constant as a function of a group's mean can be simply achieved by using a standard t-interval, based on data only from that group. The standard t-interval, however, fails to share information across the groups and is therefore not adaptive to easily obtained information about the distribution of group-specific means. In this article we construct confidence intervals that have a constant frequentist coverage rate and that make use of information about across-group heterogeneity, resulting in constant-coverage intervals that are narrower than standard t-intervals on average across groups. Such intervals are constructed by inverting biased tests for the mean of a normal population. Given a prior distribution on the mean, Bayes-optimal biased tests can be inverted to form Bayes-optimal confidence intervals with frequentist coverage that is constant as a function of the mean. In the context of multiple groups, the prior distribution is replaced by a model of across-group heterogeneity. The parameters for this model can be estimated using data from all of the groups, and used to obtain confidence intervals with constant group-specific coverage that adapt to information about the distribution of group means.