# Hypernetworks for Dynamic Feature Selection

arXiv: https://arxiv.org/abs/2605.12278 · HTML: https://mlsanity.com/paper/2605.12278/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 60 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Spectrally-normalized margin bounds for neural networks](https://mlsanity.com/paper/1706.08498/) (2017)
- [Conflict-Averse Gradient Descent for Multi-task Learning](https://mlsanity.com/paper/2110.14048/) (2021)
- [Rotation Equivariant CNNs for Digital Pathology](https://mlsanity.com/paper/1806.03962/) (2018)
- [Continual learning with hypernetworks](https://mlsanity.com/paper/1906.00695/) (2019)
- [Gradient Matching for Domain Generalization](https://mlsanity.com/paper/2104.09937/) (2021)
- [Challenges in Deploying Machine Learning: a Survey of Case Studies](https://mlsanity.com/paper/2011.09926/) (2020)
- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- [Bayesian Hypernetworks](https://mlsanity.com/paper/1710.04759/) (2017)
- [Learning to Maximize Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2301.00557/) (2023)
- [Hypernetworks in Meta-Reinforcement Learning](https://mlsanity.com/paper/2210.11348/) (2022)
- [Classification with Costly Features using Deep Reinforcement Learning](https://mlsanity.com/paper/1711.07364/) (2017)
- [Variational Information Pursuit for Interpretable Predictions](https://mlsanity.com/paper/2302.02876/) (2023)
- [HyperFast: Instant Classification for Tabular Data](https://mlsanity.com/paper/2402.14335/) (2024)
- [Estimating Conditional Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2306.03301/) (2023)
- [Acquisition Conditioned Oracle for Nongreedy Active Feature Acquisition](https://mlsanity.com/paper/2302.13960/) (2023)
- [Principled Weight Initialization for Hypernetworks](https://mlsanity.com/paper/2312.08399/) (2023)
- [HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworks](https://mlsanity.com/paper/2205.15745/) (2022)
- [Recurrent Hypernetworks are Surprisingly Strong in Meta-RL](https://mlsanity.com/paper/2309.14970/) (2023)
- [Magnitude Invariant Parametrizations Improve Hypernetwork Learning](https://mlsanity.com/paper/2304.07645/) (2023)
- [Imputation for prediction: beware of diminishing returns](https://mlsanity.com/paper/2407.19804/) (2024)
- [Stochastic Encodings for Active Feature Acquisition](https://mlsanity.com/paper/2508.01957/) (2025)

## Abstract (arXiv)

Dynamic feature selection (DFS) is a machine learning framework in which features are acquired sequentially for individual samples under budget constraints. The exponential growth in the number of possible feature acquisition paths forces a DFS model to balance fitting specific scenarios against maintaining general performance, even when the feature space is moderate in size. In this paper, we study the structural limitations of existing DFS approaches to achieve an optimal solution. Then, we propose textscHyper-DFS, a hypernetwork-based DFS approach that generates feature subset-specific classifier parameters on demand. We show that the use of hypernetworks compared to mask-embedding methods results in a smaller structural complexity bound. We also use a Set Transformer encoding to create a smooth conditioning space for the hypernetwork, so that functionally similar tasks are also geometrically close. In our benchmarks, textscHyper-DFS performed best or second best compared to all state-of-the-art approaches on synthetic and real-life tabular data. It is also best or second best across all image datasets tested, and shows stronger zero-shot generalisation to feature subsets never seen during training than existing DFS approaches. Code available in: https://github.com/Fuminides/hyperDFS