mlsanityData to 7 Oct 2026

Sparse-Input Neural Networks for High-dimensional Nonparametric Regression and Classification

arXiv 1711.07592 ↗First posted 21 Nov 2017stat.ME stat.MLcited by 39

Not on the map. Why?

Abstract (arXiv)

Neural networks are usually not the tool of choice for nonparametric high-dimensional problems where the number of input features is much larger than the number of observations. Though neural networks can approximate complex multivariate functions, they generally require a large number of training observations to obtain reasonable fits, unless one can learn the appropriate network structure. In this manuscript, we show that neural networks can be applied successfully to high-dimensional settings if the true function falls in a low dimensional subspace, and proper regularization is used. We propose fitting a neural network with a sparse group lasso penalty on the first-layer input weights. This results in a neural net that only uses a small subset of the original features. In addition, we characterize the statistical convergence of the penalized empirical risk minimizer to the optimal neural network: we show that the excess risk of this penalized estimator only grows with the logarithm of the number of input features; and we show that the weights of irrelevant features converge to zero. Via simulation studies and data analyses, we show that these sparse-input neural networks outperform existing nonparametric high-dimensional estimation methods when the data has complex higher-order interactions.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Nov 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 11; 2023: 9; 2024: 5; 2025: 3; 2026: 3.

39 papers.

Sort:

By area:

  1. 2023Learning to Maximize Mutual Information for Dynamic Feature Selectioncited by 54
  2. 2021Locally Sparse Neural Networks for Tabular Biomedical Datacited by 34
  3. 2021Consistent Sparse Deep Learning: Theory and Computationcited by 27
  4. 2024Exploring Large Language Models for Feature Selection: A Data-centric Perspectivecited by 27
  5. 2024Explaining Time Series via Contrastive and Locally Sparse Perturbationscited by 22
  6. 2023Estimating Conditional Mutual Information for Dynamic Feature Selectioncited by 20
  7. 2021How to Find a Good Explanation for Clustering?cited by 16
  8. 2022Statistical guarantees for sparse deep learningcited by 9
  9. 2021Sparse Deep Learning: A New Framework Immune to Local Traps and Miscalibrationcited by 7
  10. 2022Conditional Distribution Function Estimation Using Neural Networks for Censored and Uncensored Datacited by 7
  11. 2022GCondNet: A Novel Method for Improving Neural Networks on Small High-Dimensional Tabular Datacited by 6
  12. 2022Weight Predictor Network with Feature Selection for Small Sample Tabular Biomedical Datacited by 6
27 more
  1. 2023The Contextual Lasso: Sparse Linear Models via Deep Neural Networkscited by 6
  2. 2021Regularization and Reparameterization Avoid Vanishing Gradients in Sigmoid-Type Networkscited by 5
  3. 2022An Inexact Augmented Lagrangian Algorithm for Training Leaky ReLU Neural Network with Group Sparsitycited by 5
  4. 2025Representation Retrieval Learning for Heterogeneous Data Integrationcited by 5
  5. 2022Towards a Unified Framework for Uncertainty-aware Nonlinear Variable Selection with Theoretical Guaranteescited by 4
  6. 2022DisPositioNet: Disentangled Pose and Identity in Semantic Image Manipulationcited by 4
  7. 2023Sparse-Input Neural Network using Group Concave Regularizationcited by 4
  8. 2023Learning Channel Importance for High Content Imaging with Interpretable Deep Input Channel Mixingcited by 3
  9. 2023Feature Selection in the Contrastive Analysis Settingcited by 3
  10. 2022A phase transition for finding needles in nonlinear haystacks with LASSO artificial neural networkscited by 2
  11. 2025Regularization can make diffusion models more efficientcited by 2
  12. 2021Lockout: Sparse Regularization of Neural Networkscited by 1
  13. 2021Searching for Minimal Optimal Neural Networkscited by 1
  14. 2023SLM: End-to-end Feature Selection via Sparse Learnable Maskscited by 1
  15. 2023DeRisk: An Effective Deep Learning Framework for Credit Risk Prediction over Real-World Financial Datacited by 1
  16. 2024Penalized Generative Variable Selectioncited by 1
  17. 2025PrAda-GAN: A Private Adaptive Generative Adversarial Network with Bayes Network Structurecited by 1
  18. 2026LLM-Driven Reasoning for Constraint-Aware Feature Selection in Industrial Systemscited by 1
  19. 2021Top-k Regularization for Supervised Feature Selectioncited by 0
  20. 2022From Correlation to Causation: Formalizing Interpretable Machine Learning as a Statistical Processcited by 0
  21. 2022Error Controlled Feature Selection for Ultrahigh Dimensional and Highly Correlated Feature Space Using Deep Learningcited by 0
  22. 2022Mitigating Overfitting in Graph Neural Networks via Feature and Hyperplane Perturbationcited by 0
  23. 2023End-to-end Feature Selection Approach for Learning Skinny Treescited by 0
  24. 2024A Multi-Domain Multi-Task Approach for Feature Selection from Bulk RNA Datasetscited by 0
  25. 2024Sparse deep neural networks for nonparametric estimation in high-dimensional sparse regressioncited by 0
  26. 2026Deep Multitask Learning for Mixed-Type Outcomes with Shared Sparsitycited by 0
  27. 2026Robust LassoNet: Enhancing Feature Selection in Neural Networks via Robust Loss Functionscited by 0