mlsanityData to 14 Oct 2026

Statistical Test for Auto Feature Engineering by Selective Inference

arXiv 2410.19768 ↗First posted 13 Oct 2024stat.ML cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Auto Feature Engineering (AFE) plays a crucial role in developing practical machine learning pipelines by automating the transformation of raw data into meaningful features that enhance model performance. By generating features in a data-driven manner, AFE enables the discovery of important features that may not be apparent through human experience or intuition. On the other hand, since AFE generates features based on data, there is a risk that these features may be overly adapted to the data, making it essential to assess their reliability appropriately. Unfortunately, because most AFE problems are formulated as combinatorial search problems and solved by heuristic algorithms, it has been challenging to theoretically quantify the reliability of generated features. To address this issue, we propose a new statistical test for generated features by AFE algorithms based on a framework called selective inference. As a proof of concept, we consider a simple class of tree search-based heuristic AFE algorithms, and consider the problem of testing the generated features when they are used in a linear model. The proposed test can quantify the statistical significance of the generated features in the form of p-values, enabling theoretically guaranteed control of the risk of false findings.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 19 of its 42 references that are in the map. First posted on arXiv 13 Oct 2024.

Builds on

19 of 42 references.

References · 19

  1. 2013Exact post-selection inference, with application to the lassocited by 91
  2. 2019The autofeat Python Library for Automated Feature Engineering and Selectioncited by 73
  3. 2014Exact Post-Selection Inference for Sequential Regression Procedurescited by 48
  4. 2020Selective Inference for Hierarchical Clusteringcited by 39
  5. 2017Valid Inference Corrected for Outlier Removalcited by 26
  6. 2019Testing for a Change in Mean After Changepoint Detectioncited by 26
  7. 2020Computing Valid p-value for Optimal Changepoint by Selective Inference using Dynamic Programmingcited by 25
  8. 2019Computing Valid p-values for Image Segmentation by Selective Inferencecited by 24
  9. 2016Selective Inference for Group-Sparse Linear Modelscited by 21
  10. 2020Quantifying Statistical Significance of Neural Network-based Image Segmentation by Selective Inferencecited by 21
  11. 2015Selective Sequential Model Selectioncited by 17
  12. 2020SAFE: Scalable Automatic Feature Engineering Framework for Industrial Taskscited by 17
7 more
  1. 2023Valid P-Value for Deep Learning-Driven Salient Regioncited by 17
  2. 2021Conditional Selective Inference for Robust Regression and Outlier Detection using Piecewise-Linear Homotopy Continuationcited by 16
  3. 2021Fast and More Powerful Selective Inference for Sparse High-order Interaction Modelcited by 15
  4. 2021Tree-Values: selective inference for regression treescited by 15
  5. 2016Post Selection Inference with Kernelscited by 13
  6. 2014Exact Post Model Selection Inference for Marginal Screeningcited by 12
  7. 2023CAD-DA: Controllable Anomaly Detection after Domain Adaptation by Statistical Inferencecited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Quantifying Statistical Significance of Deep Nearest Neighbor Anomaly Detection via Selective Inferencecited by 3
  2. 2025Statistical Test for Saliency Maps of Graph Neural Networks via Selective Inferencecited by 1
  3. 2026Automatic Statistical Test for Rationally Expressible Algorithms by Selective Inference, with Applications to Feature Selectioncited by 0