mlsanityData to 14 Oct 2026

Nonconvex Regularization for Feature Selection in Reinforcement Learning

arXiv 2509.15652 ↗First posted 19 Sep 2025cs.LGcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

This work proposes an efficient batch algorithm for feature selection in reinforcement learning (RL) with theoretical convergence guarantees. To mitigate the estimation bias inherent in conventional regularization schemes, the first contribution extends policy evaluation within the classical least-squares temporal-difference (LSTD) framework by formulating a Bellman-residual objective regularized with the sparsity-inducing, nonconvex projected minimax concave (PMC) penalty. Owing to the weak convexity of the PMC penalty, this formulation can be interpreted as a special instance of a general nonmonotone-inclusion problem. The second contribution establishes novel convergence conditions for the forward-reflected-backward splitting (FRBS) algorithm to solve this class of problems. Numerical experiments on benchmark datasets demonstrate that the proposed approach substantially outperforms state-of-the-art feature-selection methods, particularly in scenarios with many noisy features.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 34 references that are in the map. First posted on arXiv 19 Sep 2025.

Builds on

9 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Nonparametric Bellman Mappings for Reinforcement Learning: Application to Robust Adaptive Filtering29 Mar

Other references · 8

  1. 2018A Forward-Backward Splitting Method for Monotone Inclusions Without Cocoercivitycited by 36
  2. 2019Generalized monotone operators and their averaged resolventscited by 27
  3. 2018Sparse Regularization via Convex Analysiscited by 17
  4. 2014Sparse Recovery via Differential Inclusionscited by 11
  5. 2015On the Convergence of the Iterative Shrinkage/Thresholding Algorithm With a Weakly Convex Penaltycited by 6
  6. 2019Linearly Involved Generalized Moreau Enhanced Models and Their Proximal Splitting Algorithm under Overall Convexity Conditioncited by 5
  7. 2024Monotone Lipschitz-Gradient Denoiser: Explainability of Operator Regularization Approaches Free From Lipschitz Constant Controlcited by 3
  8. 2022Linearly-involved Moreau-Enhanced-over-Subspace Model: Debiased Sparse Modeling and Stable Outlier-Robust Regressioncited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Non-Convex Sparse Reinforcement Learning via Non-Monotone Inclusionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Sparse Feature Selection Makes Batch Reinforcement Learning More Sample Efficient
  2. 2025Sparse-Reg: Improving Sample Complexity in Offline Reinforcement Learning using Sparsity
  3. 2023A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning
  4. 2024Optimal Policy Sparsification and Low Rank Decomposition for Deep Reinforcement Learning
  5. 2025On the Effect of Regularization in Policy Mirror Descent
  6. 2025Learning to Trust Bellman Updates: Selective State-Adaptive Regularization for Offline RL
  7. 2026A Unified and Constrained View of Regularization-Based Robust Reinforcement Learning
  8. 2026Mathematical methods of reinforcement learning