# Weight Predictor Network with Feature Selection for Small Sample Tabular Biomedical Data

arXiv: https://arxiv.org/abs/2211.15616 · HTML: https://mlsanity.com/paper/2211.15616/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 48 references that are in the map.
- First posted on arXiv 28 Nov 2022.

## Builds on (most cited first, 18 of 18)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [Neural Oblivious Decision Ensembles for Deep Learning on Tabular Data](https://mlsanity.com/paper/1909.06312/) (2019)
- [Well-tuned Simple Nets Excel on Tabular Datasets](https://mlsanity.com/paper/2106.11189/) (2021)
- [Concrete Autoencoders for Differentiable Feature Selection and Reconstruction](https://mlsanity.com/paper/1901.09346/) (2019)
- [Deep Neural Decision Trees](https://mlsanity.com/paper/1806.06988/) (2018)
- [The Tree Ensemble Layer: Differentiability meets Conditional Computation](https://mlsanity.com/paper/2002.07772/) (2020)
- [Sparse-Input Neural Networks for High-dimensional Nonparametric Regression and Classification](https://mlsanity.com/paper/1711.07592/) (2017)
- [Locally Sparse Neural Networks for Tabular Biomedical Data](https://mlsanity.com/paper/2106.06468/) (2021)
- [FsNet: Feature Selection Network on High-dimensional Biological Data](https://mlsanity.com/paper/2001.08322/) (2020)
- [Diet Networks: Thin Parameters for Fat Genomics](https://mlsanity.com/paper/1611.09340/) (2016)

## Built on by (most cited first, 6 of 6)

- [ProtoGate: Prototype-based Neural Networks with Global-to-local Feature Selection for Tabular Biomedical Data](https://mlsanity.com/paper/2306.12330/) (2023)
- [TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Models](https://mlsanity.com/paper/2409.16118/) (2024)
- [TabMDA: Tabular Manifold Data Augmentation for Any Classifier using Transformers with In-context Subsetting](https://mlsanity.com/paper/2406.01805/) (2024)
- [Toward Real-World Table Agents: Capabilities, Workflows, and Design Principles for LLM-based Table Intelligence](https://mlsanity.com/paper/2507.10281/) (2025)
- [Enhancing Representation Learning on High-Dimensional, Small-Size Tabular Data: A Divide and Conquer Method with Ensembled VAEs](https://mlsanity.com/paper/2306.15661/) (2023)
- [A Novel GAN Approach to Augment Limited Tabular Data for Short-Term Substance Use Prediction](https://mlsanity.com/paper/2407.13047/) (2024)

## Abstract (arXiv)

Tabular biomedical data is often high-dimensional but with a very small number of samples. Although recent work showed that well-regularised simple neural networks could outperform more sophisticated architectures on tabular data, they are still prone to overfitting on tiny datasets with many potentially irrelevant features. To combat these issues, we propose Weight Predictor Network with Feature Selection (WPFS) for learning neural networks from high-dimensional and small sample data by reducing the number of learnable parameters and simultaneously performing feature selection. In addition to the classification network, WPFS uses two small auxiliary networks that together output the weights of the first layer of the classification model. We evaluate on nine real-world biomedical datasets and demonstrate that WPFS outperforms other standard as well as more recent methods typically applied to tabular data. Furthermore, we investigate the proposed feature selection mechanism and show that it improves performance while providing useful insights into the learning task.