# Statistical Test for Auto Feature Engineering by Selective Inference

arXiv: https://arxiv.org/abs/2410.19768 · HTML: https://mlsanity.com/paper/2410.19768/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 42 references that are in the map.
- First posted on arXiv 13 Oct 2024.

## Builds on (most cited first, 19 of 19)

- [Exact post-selection inference, with application to the lasso](https://mlsanity.com/paper/1311.6238/) (2013)
- [The autofeat Python Library for Automated Feature Engineering and Selection](https://mlsanity.com/paper/1901.07329/) (2019)
- [Exact Post-Selection Inference for Sequential Regression Procedures](https://mlsanity.com/paper/1401.3889/) (2014)
- [Selective Inference for Hierarchical Clustering](https://mlsanity.com/paper/2012.02936/) (2020)
- [Valid Inference Corrected for Outlier Removal](https://mlsanity.com/paper/1711.10635/) (2017)
- [Testing for a Change in Mean After Changepoint Detection](https://mlsanity.com/paper/1910.04291/) (2019)
- [Computing Valid p-value for Optimal Changepoint by Selective Inference using Dynamic Programming](https://mlsanity.com/paper/2002.09132/) (2020)
- [Computing Valid p-values for Image Segmentation by Selective Inference](https://mlsanity.com/paper/1906.00629/) (2019)
- [Selective Inference for Group-Sparse Linear Models](https://mlsanity.com/paper/1607.08211/) (2016)
- [Quantifying Statistical Significance of Neural Network-based Image Segmentation by Selective Inference](https://mlsanity.com/paper/2010.01823/) (2020)
- [Selective Sequential Model Selection](https://mlsanity.com/paper/1512.02565/) (2015)
- [SAFE: Scalable Automatic Feature Engineering Framework for Industrial Tasks](https://mlsanity.com/paper/2003.02556/) (2020)
- [Valid P-Value for Deep Learning-Driven Salient Region](https://mlsanity.com/paper/2301.02437/) (2023)
- [Conditional Selective Inference for Robust Regression and Outlier Detection using Piecewise-Linear Homotopy Continuation](https://mlsanity.com/paper/2104.10840/) (2021)
- [Fast and More Powerful Selective Inference for Sparse High-order Interaction Model](https://mlsanity.com/paper/2106.04929/) (2021)
- [Tree-Values: selective inference for regression trees](https://mlsanity.com/paper/2106.07816/) (2021)
- [Post Selection Inference with Kernels](https://mlsanity.com/paper/1610.03725/) (2016)
- [Exact Post Model Selection Inference for Marginal Screening](https://mlsanity.com/paper/1402.5596/) (2014)
- [CAD-DA: Controllable Anomaly Detection after Domain Adaptation by Statistical Inference](https://mlsanity.com/paper/2310.14608/) (2023)

## Built on by (most cited first, 3 of 3)

- [Quantifying Statistical Significance of Deep Nearest Neighbor Anomaly Detection via Selective Inference](https://mlsanity.com/paper/2502.12978/) (2025)
- [Statistical Test for Saliency Maps of Graph Neural Networks via Selective Inference](https://mlsanity.com/paper/2505.16893/) (2025)
- [Automatic Statistical Test for Rationally Expressible Algorithms by Selective Inference, with Applications to Feature Selection](https://mlsanity.com/paper/2608.04667/) (2026)

## Abstract (arXiv)

Auto Feature Engineering (AFE) plays a crucial role in developing practical machine learning pipelines by automating the transformation of raw data into meaningful features that enhance model performance. By generating features in a data-driven manner, AFE enables the discovery of important features that may not be apparent through human experience or intuition. On the other hand, since AFE generates features based on data, there is a risk that these features may be overly adapted to the data, making it essential to assess their reliability appropriately. Unfortunately, because most AFE problems are formulated as combinatorial search problems and solved by heuristic algorithms, it has been challenging to theoretically quantify the reliability of generated features. To address this issue, we propose a new statistical test for generated features by AFE algorithms based on a framework called selective inference. As a proof of concept, we consider a simple class of tree search-based heuristic AFE algorithms, and consider the problem of testing the generated features when they are used in a linear model. The proposed test can quantify the statistical significance of the generated features in the form of p-values, enabling theoretically guaranteed control of the risk of false findings.