# Nonconvex Regularization for Feature Selection in Reinforcement Learning

arXiv: https://arxiv.org/abs/2509.15652 · HTML: https://mlsanity.com/paper/2509.15652/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 34 references that are in the map.
- First posted on arXiv 19 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 9 of 9)

- [A Forward-Backward Splitting Method for Monotone Inclusions Without Cocoercivity](https://mlsanity.com/paper/1808.04162/) (2018)
- [Generalized monotone operators and their averaged resolvents](https://mlsanity.com/paper/1902.09827/) (2019)
- [Sparse Regularization via Convex Analysis](https://mlsanity.com/paper/1803.06765/) (2018)
- [Sparse Recovery via Differential Inclusions](https://mlsanity.com/paper/1406.7728/) (2014)
- [Nonparametric Bellman Mappings for Reinforcement Learning: Application to Robust Adaptive Filtering](https://mlsanity.com/paper/2403.20020/) (2024)
- [On the Convergence of the Iterative Shrinkage/Thresholding Algorithm With a Weakly Convex Penalty](https://mlsanity.com/paper/1510.07821/) (2015)
- [Linearly Involved Generalized Moreau Enhanced Models and Their Proximal Splitting Algorithm under Overall Convexity Condition](https://mlsanity.com/paper/1910.10337/) (2019)
- [Monotone Lipschitz-Gradient Denoiser: Explainability of Operator Regularization Approaches Free From Lipschitz Constant Control](https://mlsanity.com/paper/2406.04676/) (2024)
- [Linearly-involved Moreau-Enhanced-over-Subspace Model: Debiased Sparse Modeling and Stable Outlier-Robust Regression](https://mlsanity.com/paper/2201.03235/) (2022)

## Built on by (most cited first, 1 of 1)

- [Non-Convex Sparse Reinforcement Learning via Non-Monotone Inclusions](https://mlsanity.com/paper/2607.04990/) (2026)

## Abstract (arXiv)

This work proposes an efficient batch algorithm for feature selection in reinforcement learning (RL) with theoretical convergence guarantees. To mitigate the estimation bias inherent in conventional regularization schemes, the first contribution extends policy evaluation within the classical least-squares temporal-difference (LSTD) framework by formulating a Bellman-residual objective regularized with the sparsity-inducing, nonconvex projected minimax concave (PMC) penalty. Owing to the weak convexity of the PMC penalty, this formulation can be interpreted as a special instance of a general nonmonotone-inclusion problem. The second contribution establishes novel convergence conditions for the forward-reflected-backward splitting (FRBS) algorithm to solve this class of problems. Numerical experiments on benchmark datasets demonstrate that the proposed approach substantially outperforms state-of-the-art feature-selection methods, particularly in scenarios with many noisy features.