# A parametric distribution for exact post-selection inference with data carving

arXiv: https://arxiv.org/abs/2305.12581 · HTML: https://mlsanity.com/paper/2305.12581/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 21 May 2023.

## Builds on (most cited first, 11 of 11)

- [Exact post-selection inference, with application to the lasso](https://mlsanity.com/paper/1311.6238/) (2013)
- [Optimal Inference After Model Selection](https://mlsanity.com/paper/1410.2597/) (2014)
- [Exact Post-Selection Inference for Sequential Regression Procedures](https://mlsanity.com/paper/1401.3889/) (2014)
- [False Discoveries Occur Early on the Lasso Path](https://mlsanity.com/paper/1511.01957/) (2015)
- [A significance test for the lasso](https://mlsanity.com/paper/1301.7161/) (2013)
- [Exact Post Model Selection Inference for Marginal Screening](https://mlsanity.com/paper/1402.5596/) (2014)
- [A Study of Error Variance Estimation in Lasso Regression](https://mlsanity.com/paper/1311.5274/) (2013)
- [Unifying approach to selective inference with applications to cross-validation](https://mlsanity.com/paper/1703.06559/) (2017)
- [Multicarving for high-dimensional post-selection inference](https://mlsanity.com/paper/2006.04613/) (2020)
- [A (tight) upper bound for the length of confidence intervals with conditional coverage](https://mlsanity.com/paper/2007.12448/) (2020)
- [Selective inference after cross-validation](https://mlsanity.com/paper/1511.08866/) (2015)

## Abstract (arXiv)

Post-selection inference (PoSI) is a statistical technique for obtaining valid confidence intervals and p-values when hypothesis generation and testing use the same source of data. PoSI can be used on a range of popular algorithms including the Lasso. Data carving is a variant of PoSI in which a portion of held out data is combined with the hypothesis generating data at inference time. While data carving has attractive theoretical and empirical properties, existing approaches rely on computationally expensive MCMC methods to carry out inference. This paper's key contribution is to show that pivotal quantities can be constructed for the data carving procedure based on a known parametric distribution. Specifically, when the selection event is characterized by a set of polyhedral constraints on a Gaussian response, data carving will follow the sum of a normal and a truncated normal (SNTN), which is a variant of the truncated bivariate normal distribution. The main impact of this insight is that obtaining exact inference for data carving can be made computationally trivial, since the CDF of the SNTN distribution can be found using the CDF of a standard bivariate normal. A python package sntn has been released to further facilitate the adoption of data carving with PoSI.