# Post-ADC Inference: Valid Inference After Active Data Collection

arXiv: https://arxiv.org/abs/2605.11511 · HTML: https://mlsanity.com/paper/2605.11511/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 29 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 7 of 7)

- [Optimal Inference After Model Selection](https://mlsanity.com/paper/1410.2597/) (2014)
- [Exact Post-Selection Inference for Sequential Regression Procedures](https://mlsanity.com/paper/1401.3889/) (2014)
- [More powerful post-selection inference, with application to the Lasso](https://mlsanity.com/paper/1801.09037/) (2018)
- [Selective Sequential Model Selection](https://mlsanity.com/paper/1512.02565/) (2015)
- [Unifying approach to selective inference with applications to cross-validation](https://mlsanity.com/paper/1703.06559/) (2017)
- [Optimal Conditional Inference in Adaptive Experiments](https://mlsanity.com/paper/2309.12162/) (2023)
- [Selective Randomization Inference for Adaptive Experiments](https://mlsanity.com/paper/2405.07026/) (2024)

## Abstract (arXiv)

The validity of statistical inference depends critically on how data are collected. When data gathered through active data collection (ADC) are reused for a post-hoc inferential task, conventional inference can fail because the sampling is adaptively biased toward regions favored by the collection strategy. This issue is especially pronounced in black-box optimization, where sequential model-based optimization (SMBO) methods such as the tree-structured Parzen estimator (TPE) and Gaussian process upper confidence bound (GP-UCB) preferentially concentrate evaluations in promising regions. We study statistical inference on actively collected data when the inferential target is constructed in a data-dependent manner after data collection. To enable valid inference in this setting, we propose post-ADC inference, a framework that accounts for the biases arising from both the active data collection process and the subsequent data-driven target construction. Our method builds on selective inference and provides valid p-values and confidence intervals that correct for both sources of bias. The framework applies to a broad class of ADC processes by imposing only assumptions on the observation noise, without requiring any assumptions on the underlying black-box function or the surrogate model used by the SMBO algorithm. Empirical results also show that post-ADC inference provides valid inference for data collected by GP-UCB and TPE.