# Non-Myopic Active Feature Acquisition via Pathwise Policy Gradients

arXiv: https://arxiv.org/abs/2605.05511 · HTML: https://mlsanity.com/paper/2605.05511/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 30 of its 63 references that are in the map.
- First posted on arXiv 6 May 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 30 of 30)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [PonderNet: Learning to Ponder](https://mlsanity.com/paper/2107.05407/) (2021)
- [Learning Continuous Control Policies by Stochastic Value Gradients](https://mlsanity.com/paper/1510.09142/) (2015)
- [EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE](https://mlsanity.com/paper/1809.11142/) (2018)
- [Partially Observable Markov Decision Processes in Robotics: A Survey](https://mlsanity.com/paper/2209.10342/) (2022)
- [Learning to Maximize Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2301.00557/) (2023)
- [Active Feature Acquisition with Generative Surrogate Models](https://mlsanity.com/paper/2010.02433/) (2020)
- [Online Planning Algorithms for POMDPs](https://mlsanity.com/paper/1401.3436/) (2014)
- [Learning to Stop While Learning to Predict](https://mlsanity.com/paper/2006.05082/) (2020)
- [Variational Information Pursuit for Interpretable Predictions](https://mlsanity.com/paper/2302.02876/) (2023)
- [Classification with Costly Features as a Sequential Decision-Making Problem](https://mlsanity.com/paper/1909.02564/) (2019)
- [Estimating Conditional Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2306.03301/) (2023)
- [Opportunistic Learning: Budgeted Cost-Sensitive Learning from Data Streams](https://mlsanity.com/paper/1901.00243/) (2019)
- [Acquisition Conditioned Oracle for Nongreedy Active Feature Acquisition](https://mlsanity.com/paper/2302.13960/) (2023)
- [A Survey on Active Feature Acquisition Strategies](https://mlsanity.com/paper/2502.11067/) (2025)
- [Stochastic Encodings for Active Feature Acquisition](https://mlsanity.com/paper/2508.01957/) (2025)
- [AFABench: A Generic Framework for Benchmarking Active Feature Acquisition](https://mlsanity.com/paper/2508.14734/) (2025)
- [Efficient Online Learning for Optimizing Value of Information: Theory and Application to Interactive Troubleshooting](https://mlsanity.com/paper/1703.05452/) (2017)
- [Optimal Value of Information in Graphical Models](https://mlsanity.com/paper/1401.3474/) (2014)
- [Model-Based Reparameterization Policy Gradient Methods: Theory and Practical Algorithms](https://mlsanity.com/paper/2310.19927/) (2023)
- [Relative Entropy Pathwise Policy Optimization](https://mlsanity.com/paper/2507.11019/) (2025)
- [The Curious Language Model: Strategic Test-Time Information Acquisition](https://mlsanity.com/paper/2506.09173/) (2025)
- [Bias-Variance Tradeoffs in Single-Sample Binary Gradient Estimators](https://mlsanity.com/paper/2110.03549/) (2021)
- [Distribution Guided Active Feature Acquisition](https://mlsanity.com/paper/2410.03915/) (2024)

## Built on by (most cited first, 1 of 1)

- [Active Feature Acquisition With Incomplete Training Data](https://mlsanity.com/paper/2609.32325/) (2026)

## Abstract (arXiv)

Active feature acquisition (AFA) considers prediction problems in which features are costly to obtain and the learner adaptively decides which feature values to acquire for each instance and when to stop and predict. In this paper, we introduce a continuous relaxation of the acquisition process that enables non-myopic pathwise policy gradients (NM-PPG) through the full acquisition trajectory, avoiding the high variance of standard score-function policy gradients while allowing end-to-end optimization of the acquisition policy. To better align training with deployment, we develop a straight-through rollout that follows discrete feature acquisitions in the forward pass while backpropagating through the corresponding soft relaxation. We derive an AFA-specific average-case upper bound on the variance of this gradient estimator, which characterizes instability and motivates staged temperature sharpening. Experiments on both synthetic and real-world datasets demonstrate that NM-PPG yields superior performance relative to state-of-the-art AFA baselines.