# Classification with Costly Features using Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1711.07364 · HTML: https://mlsanity.com/paper/1711.07364/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2017.

## Built on by (most cited first, 30 of 34)

- [Learning to Maximize Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2301.00557/) (2023)
- [Deep Reinforcement Learning for Cost-Effective Medical Diagnosis](https://mlsanity.com/paper/2302.10261/) (2023)
- [Estimating Conditional Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2306.03301/) (2023)
- [A Survey on Active Feature Acquisition Strategies](https://mlsanity.com/paper/2502.11067/) (2025)
- [Active Acquisition for Multimodal Temporal Data: A Challenging Decision-Making Task](https://mlsanity.com/paper/2211.05039/) (2022)
- [Clairvoyance: A Pipeline Toolkit for Medical Time Series](https://mlsanity.com/paper/2310.18688/) (2023)
- [Stochastic Encodings for Active Feature Acquisition](https://mlsanity.com/paper/2508.01957/) (2025)
- [DRMD: Deep Reinforcement Learning for Malware Detection under Concept Drift](https://mlsanity.com/paper/2508.18839/) (2025)
- [Towards Trustworthy Automatic Diagnosis Systems by Emulating Doctors' Reasoning with Deep Reinforcement Learning](https://mlsanity.com/paper/2210.07198/) (2022)
- [Learning-To-Measure: In-Context Active Feature Acquisition](https://mlsanity.com/paper/2510.12624/) (2025)
- [Costly Features Classification using Monte Carlo Tree Search](https://mlsanity.com/paper/2102.07073/) (2021)
- [Mapping the Internet: Modelling Entity Interactions in Complex Heterogeneous Networks](https://mlsanity.com/paper/2104.09650/) (2021)
- [RLAS-BIABC: A Reinforcement Learning-Based Answer Selection Using the BERT Model Boosted by an Improved ABC Algorithm](https://mlsanity.com/paper/2301.02807/) (2023)
- [Deep Reinforcement Learning for Personalized Diagnostic Decision Pathways Using Electronic Health Records: A Comparative Study on Anemia and Systemic Lupus Erythematosus](https://mlsanity.com/paper/2404.05913/) (2024)
- [Learning Computational Efficient Bots with Costly Features](https://mlsanity.com/paper/2308.09629/) (2023)
- [Generator Assisted Mixture of Experts For Feature Acquisition in Batch](https://mlsanity.com/paper/2312.12574/) (2023)
- [Dynamic Instance-Wise Classification in Correlated Feature Spaces](https://mlsanity.com/paper/2106.04668/) (2021)
- [UnfoldML: Cost-Aware and Uncertainty-Based Dynamic 2D Prediction for Multi-Stage Classification](https://mlsanity.com/paper/2210.15056/) (2022)
- [Fast Classification with Sequential Feature Selection in Test Phase](https://mlsanity.com/paper/2306.14347/) (2023)
- [Dynamic Feature Selection from Variable Feature Sets Using Features of Features](https://mlsanity.com/paper/2503.09181/) (2025)
- [Model-Agnostic Dynamic Feature Selection with Uncertainty Quantification](https://mlsanity.com/paper/2508.02566/) (2025)
- [When Should Humans Step In? Optimal Human Dispatching in AI-Assisted Decisions](https://mlsanity.com/paper/2603.13688/) (2026)
- [Scalable Online Disease Diagnosis via Multi-Model-Fused Actor-Critic Reinforcement Learning](https://mlsanity.com/paper/2206.03659/) (2022)
- [Extracting Diagnosis Pathways from Electronic Health Records Using Deep Reinforcement Learning](https://mlsanity.com/paper/2305.06295/) (2023)
- [EEG_RL-Net: Enhancing EEG MI Classification through Reinforcement Learning-Optimised Graph Neural Networks](https://mlsanity.com/paper/2405.00723/) (2024)
- [AutoNFS: Automatic Neural Feature Selection](https://mlsanity.com/paper/2503.13304/) (2025)
- [DARTS: Targeting Prognostic Covariates in Budget-Constrained Sequential Experiments](https://mlsanity.com/paper/2605.06608/) (2026)
- [Hypernetworks for Dynamic Feature Selection](https://mlsanity.com/paper/2605.12278/) (2026)
- [Zero-Shot Active Feature Acquisition via LLM-Elicitation](https://mlsanity.com/paper/2606.18933/) (2026)
- [Evaluator-Dependent Patient-Adaptive ECG Lead-Channel Allocation](https://mlsanity.com/paper/2608.26827/) (2026)

## Abstract (arXiv)

We study a classification problem where each feature can be acquired for a cost and the goal is to optimize a trade-off between the expected classification error and the feature cost. We revisit a former approach that has framed the problem as a sequential decision-making problem and solved it by Q-learning with a linear approximation, where individual actions are either requests for feature values or terminate the episode by providing a classification decision. On a set of eight problems, we demonstrate that by replacing the linear approximation with neural networks the approach becomes comparable to the state-of-the-art algorithms developed specifically for this problem. The approach is flexible, as it can be improved with any new reinforcement learning enhancement, it allows inclusion of pre-trained high-performance classifier, and unlike prior art, its performance is robust across all evaluated datasets.