mlsanityData to 7 Oct 2026

Learning to Maximize Mutual Information for Dynamic Feature Selection

arXiv 2301.00557 ↗First posted 2 Jan 2023cs.LG cs.IT math.ITcited by 54

Not on the map. Why?

Abstract (arXiv)

Feature selection helps reduce data acquisition costs in ML, but the standard approach is to train models with static feature subsets. Here, we consider the dynamic feature selection (DFS) problem where a model sequentially queries features based on the presently available information. DFS is often addressed with reinforcement learning, but we explore a simpler approach of greedily selecting features based on their conditional mutual information. This method is theoretically appealing but requires oracle access to the data distribution, so we develop a learning approach based on amortized optimization. The proposed method is shown to recover the greedy policy when trained to optimality, and it outperforms numerous existing feature selection methods in our experiments, thus validating it as a simple but powerful approach for this problem.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 27 of its 72 references that are in the map. First posted on arXiv 2 Jan 2023.

Builds on

27 of 72 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  5. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  6. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  7. 2017Deep Reinforcement Learning that Matterscited by 727
  8. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  9. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  10. 2014Recurrent Models of Visual Attentioncited by 344
  11. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  12. 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
15 more
  1. 2018Handling Incomplete Heterogeneous Data using VAEscited by 148
  2. 2017Backpropagation through the Void: Optimizing control variates for black-box gradient estimationcited by 104
  3. 2019Concrete Autoencoders for Differentiable Feature Selection and Reconstructioncited by 90
  4. 2018EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAEcited by 85
  5. 2020VAEM: a Deep Generative Model for Heterogeneous Mixed Type Datacited by 66
  6. 2021Have We Learned to Explain?: How Interpretability Methods Can Learn to Encode Predictions in their Interpretationscited by 59
  7. 2020Active Feature Acquisition with Generative Surrogate Modelscited by 53
  8. 2016Restricted Strong Convexity Implies Weak Submodularitycited by 51
  9. 2017Sparse-Input Neural Networks for High-dimensional Nonparametric Regression and Classificationcited by 39
  10. 2017Classification with Costly Features using Deep Reinforcement Learningcited by 32
  11. 2022Interpretable by Design: Learning Predictors by Composing Interpretable Queriescited by 26
  12. 2019Opportunistic Learning: Budgeted Cost-Sensitive Learning from Data Streamscited by 18
  13. 2017Dropout Feature Ranking for Deep Learning Modelscited by 13
  14. 2016Active Information Acquisitioncited by 11
  15. 2016Dynamic Question Ordering in Online Surveyscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 12; 2025: 17; 2026: 19.

54 papers.

Sort:

  1. 2024LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Textscited by 81
  2. 2024Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attributioncited by 21
  3. 2023Estimating Conditional Mutual Information for Dynamic Feature Selectioncited by 20
  4. 2025A Survey on Active Feature Acquisition Strategiescited by 16
  5. 2026Don't Always Pick the Highest-Performing Model: An Information Theoretic View of LLM Ensemble Selectioncited by 10
  6. 2024All models are wrong, some are useful: Model Selection with Limited Labelscited by 8
  7. 2025Stochastic Encodings for Active Feature Acquisitioncited by 8
  8. 2025Conformal Information Pursuit for Interactively Guiding Large Language Modelscited by 7
  9. 2025AFABench: A Generic Framework for Benchmarking Active Feature Acquisitioncited by 7
  10. 2023Contextual Feature Selection with Conditional Stochastic Gatescited by 6
  11. 2024Information-Theoretic Distillation for Reference-less Summarizationcited by 6
  12. 2025Learning-To-Measure: In-Context Active Feature Acquisitioncited by 4
38 more
  1. 2023Feature Selection in the Contrastive Analysis Settingcited by 3
  2. 2024TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learningcited by 3
  3. 2024Constrained Multiview Representation for Self-supervised Contrastive Learningcited by 3
  4. 2024Partial Information Decomposition for Data Interpretability and Feature Selectioncited by 3
  5. 2024Feature Selection for Network Intrusion Detectioncited by 3
  6. 2024MINT: A wrapper to make multi-modal and multi-image AI models interactivecited by 2
  7. 2025Predicting Through Generation: Why Generation Is Better for Predictioncited by 2
  8. 2025IP-CRR: Information Pursuit for Interpretable Classification of Chest Radiology Reportscited by 2
  9. 2025From Pixels to Perception: Interpretable Predictions via Instance-wise Grouped Feature Selectioncited by 2
  10. 2025PRECISE-AS: Personalized Reinforcement Learning for Efficient Point-of-Care Echocardiography in Aortic Stenosis Diagnosiscited by 2
  11. 2023A distributed neural network architecture for dynamic sensor selection with application to bandwidth-constrained body-sensor networkscited by 1
  12. 2024Local Feature Selection without Label or Feature Leakage for Interpretable Machine Learning Predictionscited by 1
  13. 2025Dynamic Feature Selection from Variable Feature Sets Using Features of Featurescited by 1
  14. 2025FRET: Feature Redundancy Elimination for Test Time Adaptationcited by 1
  15. 2025NOCTA: Non-Greedy Objective Cost-Tradeoff Acquisition for Longitudinal Datacited by 1
  16. 2025Model-Agnostic Dynamic Feature Selection with Uncertainty Quantificationcited by 1
  17. 2026Non-Myopic Active Feature Acquisition via Pathwise Policy Gradientscited by 1
  18. 2023Learning From High-Dimensional Cyber-Physical Data Streams for Diagnosing Faults in Smart Gridscited by 0
  19. 2023Variational Information Pursuit with Large Language and Multimodal Models for Interpretable Predictionscited by 0
  20. 2024Multiple-Input Auto-Encoder Guided Feature Selection for IoT Intrusion Detection Systemscited by 0
  21. 2024Towards Dynamic Feature Acquisition on Medical Time Series by Maximizing Conditional Mutual Informationcited by 0
  22. 2025A Study on the Importance of Features in Detecting Advanced Persistent Threats Using Machine Learningcited by 0
  23. 2025AutoNFS: Automatic Neural Feature Selectioncited by 0
  24. 2025Unsupervised Dynamic Feature Selection for Robust Latent Spaces in Vision Taskscited by 0
  25. 2025DynamiX: Dynamic Resource eXploration for Personalized Ad-Recommendationscited by 0
  26. 2026Aligned explanations in neural networkscited by 0
  27. 2026Ca-MCF: Category-level Multi-label Causal Feature selectioncited by 0
  28. 2026Relaxed Efficient Acquisition of Context and Temporal Featurescited by 0
  29. 2026Sampling-guided exploration of active feature selection policiescited by 0
  30. 2026Uncertainty as a Planning Signal: Multi-Turn Decision Making for Goal-Oriented Conversationcited by 0
  31. 2026DARTS: Targeting Prognostic Covariates in Budget-Constrained Sequential Experimentscited by 0
  32. 2026Hypernetworks for Dynamic Feature Selectioncited by 0
  33. 2026Bayesian Selective Latent Inference for Wastewater-First Influenza Monitoringcited by 0
  34. 2026Zero-Shot Active Feature Acquisition via LLM-Elicitationcited by 0
  35. 2026BRiG-AFA: Bellman Risk-to-Go Learning for Non-Myopic Active Feature Acquisitioncited by 0
  36. 2026Conditional Validity for Adaptive Modality Acquisition: When the Policy Chooses Its Own Calibration Groupcited by 0
  37. 2026Evaluator-Dependent Patient-Adaptive ECG Lead-Channel Allocationcited by 0
  38. 2026From Uncertainty to Clinical Risk: Severity-Aware Conformal Planning for Interactive Medical Diagnosiscited by 0

50 most cited of 54 All 54 (JSON)