# A significance test for forward stepwise model selection

arXiv: https://arxiv.org/abs/1405.3920 · HTML: https://mlsanity.com/paper/1405.3920/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2014.

## Built on by (most cited first, 17 of 17)

- [More Powerful Conditional Selective Inference for Generalized Lasso by Parametric Programming](https://mlsanity.com/paper/2105.04920/) (2021)
- [Selective inference for k-means clustering](https://mlsanity.com/paper/2203.15267/) (2022)
- [Conditional Selective Inference for Robust Regression and Outlier Detection using Piecewise-Linear Homotopy Continuation](https://mlsanity.com/paper/2104.10840/) (2021)
- [Fast and More Powerful Selective Inference for Sparse High-order Interaction Model](https://mlsanity.com/paper/2106.04929/) (2021)
- [DRCFS: Doubly Robust Causal Feature Selection](https://mlsanity.com/paper/2306.07024/) (2023)
- [Valid and Exact Statistical Inference for Multi-dimensional Multiple Change-Points by Selective Inference](https://mlsanity.com/paper/2110.08989/) (2021)
- [CAD-DA: Controllable Anomaly Detection after Domain Adaptation by Statistical Inference](https://mlsanity.com/paper/2310.14608/) (2023)
- [Exact Statistical Inference for the Wasserstein Distance by Selective Inference](https://mlsanity.com/paper/2109.14206/) (2021)
- [Statistical Test for Anomaly Detections by Variational Auto-Encoders](https://mlsanity.com/paper/2402.03724/) (2024)
- [Statistical Test for Attention Map in Vision Transformer](https://mlsanity.com/paper/2401.08169/) (2024)
- [Statistical Inference for Feature Selection after Optimal Transport-based Domain Adaptation](https://mlsanity.com/paper/2410.15022/) (2024)
- [Statistical Inference for Autoencoder-based Anomaly Detection after Representation Learning-based Domain Adaptation](https://mlsanity.com/paper/2508.07049/) (2025)
- [Bounded P-values in Parametric Programming-based Selective Inference](https://mlsanity.com/paper/2307.11351/) (2023)
- [Statistical Test for Saliency Maps of Graph Neural Networks via Selective Inference](https://mlsanity.com/paper/2505.16893/) (2025)
- [Statistical Inference for the Dynamic Time Warping Distance, with Application to Abnormal Time-Series Detection](https://mlsanity.com/paper/2202.06593/) (2022)
- [Post-Transfer Learning Statistical Inference in High-Dimensional Regression](https://mlsanity.com/paper/2504.18212/) (2025)
- [Statistical Inference for Clustering-based Anomaly Detection](https://mlsanity.com/paper/2504.18633/) (2025)

## Abstract (arXiv)

We apply the methods developed by Lockhart et al. (2013) and Taylor et al. (2013) on significance tests for penalized regression to forward stepwise model selection. A general framework for selection procedures described by quadratic inequalities includes a variant of forward stepwise with grouped variables, allowing us to handle categorical variables and factor models. We provide an algorithm to compute a new statistic with an exact null distribution conditional on the outcome of the model selection procedure. This new statistic, which we denote Tchi, has a truncated chi distribution under the global null. We apply this test in forward stepwise iteratively on the residual after each step. The resulting method has the computational strengths of stepwise selection and addresses the problem of invalid test statistics due to model selection. We illustrate the flexibility of this method by applying it to several specialized applications of forward stepwise including a hierarchical interactions model and a recently described additive model that adaptively chooses between linear and nonlinear effects for each variable.