mlsanityData to 7 Oct 2026

Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guarantees

arXiv 2204.07293 ↗First posted 15 Apr 2022stat.ML cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

We develop a simple and unified framework for nonlinear variable selection that incorporates uncertainty in the prediction function and is compatible with a wide range of machine learning models (e.g., tree ensembles, kernel methods, neural networks, etc). In particular, for a learned nonlinear model f(x), we consider quantifying the importance of an input variable xj using the integrated partial derivative Psij = Vert fracpartialpartial xj f(x)Vert2PX. We then (1) provide a principled approach for quantifying variable selection uncertainty by deriving its posterior distribution, and (2) show that the approach is generalizable even to non-differentiable models such as tree ensembles. Rigorous Bayesian nonparametric theorems are derived to guarantee the posterior consistency and asymptotic uncertainty of the proposed approach. Extensive simulations and experiments on healthcare benchmark datasets confirm that the proposed algorithm outperforms existing classic and recent variable selection methods.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 36 of its 98 references that are in the map. First posted on arXiv 15 Apr 2022.

Builds on

36 of 98 references.

References · 36

  1. 2017Deep Neural Networks as Gaussian Processescited by 617
  2. 2015Deep Kernel Learningcited by 440
  3. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  4. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  5. 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 410
  6. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  7. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  8. 2018Gaussian Process Behaviour in Wide Deep Neural Networkscited by 314
  9. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  10. 2017Distilling a Neural Network Into a Soft Decision Treecited by 230
  11. 2020A Survey on Causal Inferencecited by 202
  12. 2016Generalization Properties of Learning with Random Featurescited by 163
24 more
  1. 2016Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selectioncited by 120
  2. 2020Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyondcited by 107
  3. 2016Stochastic Variational Deep Kernel Learningcited by 92
  4. 2014Manifold Gaussian Processes for Regressioncited by 79
  5. 2018Deep Neural Decision Treescited by 72
  6. 2018Adaptive Neural Treescited by 64
  7. 2020Triple descent and the two kinds of overfitting: Where & why do they appear?cited by 64
  8. 2020Implicit Regularization of Random Feature Modelscited by 54
  9. 2018Posterior Concentration for Sparse Deep Learningcited by 50
  10. 2018DeepPINK: reproducible feature selection in deep neural networkscited by 41
  11. 2017Sparse-Input Neural Networks for High-dimensional Nonparametric Regression and Classificationcited by 39
  12. 2019Benchmarking the Neural Linear Model for Regressioncited by 31
  13. 2016Neural Random Forestscited by 25
  14. 2020Gaussian Process Boostingcited by 25
  15. 2014The Random Forest Kernel and other kernels for big data from random partitionscited by 18
  16. 2021Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descentcited by 14
  17. 2020Uncertainty Quantification for Sparse Deep Learningcited by 13
  18. 2020Trajectories, bifurcations and pseudotime in large clinical datasets: applications to myocardial infarction and diabetes datacited by 13
  19. 2016The Mondrian Kernelcited by 9
  20. 2015Bayesian Approximate Kernel Regression with Variable Selectioncited by 6
  21. 2020Uncertainty-Aware (UNA) Bases for Deep Bayesian Regression Using Multi-Headed Auxiliary Networkscited by 6
  22. 2017Random Forests, Decision Trees, and Categorical Predictors: The "Absent Levels" Problemcited by 5
  23. 2019Variable Selection with Rigorous Uncertainty Quantification using Deep Bayesian Neural Networks: Posterior Concentration and Bernstein-von Mises Phenomenoncited by 4
  24. 2019Gaussian Process Regression and Classification under Mathematical Constraints with Learning Guaranteescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Statistical Advantages of Oblique Randomized Decision Trees and Forestscited by 1
  2. 2024TrIM: Transformed Iterative Mondrian Forests for Gradient-based Dimension Reduction and High-Dimensional Regressioncited by 1
  3. 2025FALCON: Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large Language Modelcited by 1
  4. 2026Calibrated Derivative-Process Sensitivity for Gaussian-Process Variable Selectioncited by 0