# Non-Convex Sparse Reinforcement Learning via Non-Monotone Inclusions

arXiv: https://arxiv.org/abs/2607.04990 · HTML: https://mlsanity.com/paper/2607.04990/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 55 references that are in the map.
- First posted on arXiv 6 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 19 of 19)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Efficient Methods for Structured Nonconvex-Nonconcave Min-Max Optimization](https://mlsanity.com/paper/2011.00364/) (2020)
- [A Forward-Backward Splitting Method for Monotone Inclusions Without Cocoercivity](https://mlsanity.com/paper/1808.04162/) (2018)
- [On the Convergence Properties of Non-Euclidean Extragradient Methods for Variational Inequalities with Generalized Monotone Operators](https://mlsanity.com/paper/1311.2776/) (2013)
- [Generalized monotone operators and their averaged resolvents](https://mlsanity.com/paper/1902.09827/) (2019)
- [Solving Nonconvex-Nonconcave Min-Max Problems exhibiting Weak Minty Solutions](https://mlsanity.com/paper/2201.12247/) (2022)
- [Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems](https://mlsanity.com/paper/2302.09831/) (2023)
- [Sparse Regularization via Convex Analysis](https://mlsanity.com/paper/1803.06765/) (2018)
- [Beyond the Golden Ratio for Variational Inequality Algorithms](https://mlsanity.com/paper/2212.13955/) (2022)
- [Sparse Recovery via Differential Inclusions](https://mlsanity.com/paper/1406.7728/) (2014)
- [Nonparametric Bellman Mappings for Reinforcement Learning: Application to Robust Adaptive Filtering](https://mlsanity.com/paper/2403.20020/) (2024)
- [On the Convergence of the Iterative Shrinkage/Thresholding Algorithm With a Weakly Convex Penalty](https://mlsanity.com/paper/1510.07821/) (2015)
- [Linearly Involved Generalized Moreau Enhanced Models and Their Proximal Splitting Algorithm under Overall Convexity Condition](https://mlsanity.com/paper/1910.10337/) (2019)
- [Convergence of the Preconditioned Proximal Point Method and Douglas-Rachford Splitting in the Absence of Monotonicity](https://mlsanity.com/paper/2305.03605/) (2023)
- [Stochastic Variance-Reduced Forward-Reflected-Backward Splitting Methods for Nonmonotone Generalized Equations](https://mlsanity.com/paper/2406.00937/) (2024)
- [Monotone Lipschitz-Gradient Denoiser: Explainability of Operator Regularization Approaches Free From Lipschitz Constant Control](https://mlsanity.com/paper/2406.04676/) (2024)
- [Linearly-involved Moreau-Enhanced-over-Subspace Model: Debiased Sparse Modeling and Stable Outlier-Robust Regression](https://mlsanity.com/paper/2201.03235/) (2022)
- [Douglas--Rachford algorithm for nonmonotone multioperator inclusion problems](https://mlsanity.com/paper/2501.02752/) (2025)
- [Nonconvex Regularization for Feature Selection in Reinforcement Learning](https://mlsanity.com/paper/2509.15652/) (2025)

## Abstract (arXiv)

This work delivers two key contributions: one to efficient feature selection in reinforcement learning (RL), the other to the theory of non-monotone inclusions. On the RL side, the estimation bias inherent in conventional regularization schemes is addressed by augmenting classical least-squares temporal-difference (LSTD) policy evaluation with the sparsity-inducing, non-convex projected minimax concave (PMC) penalty. Because the PMC penalty is weakly convex, the resulting fixed-point problem is no longer monotone; instead, it falls under a broader class of non-monotone inclusions involving the sum of a monotone Lipschitz operator and a hypomonotone operator. On the theory side, novel convergence conditions are developed for the forward-reflected-backward splitting (FRBS) method applied to this broader class of non-monotone inclusion problems. Under mild conditions, Lyapunov stability and the existence of a limit point of the sequence of FRBS iterates are established; alternatively, under the weak Minty variational inequality assumption, exact convergence is guaranteed. Numerical tests on benchmark datasets show that the proposed FRBS iterates, applied to the non-convexly regularized LSTD problem, substantially outperform state-of-the-art feature-selection methods, especially when many noisy features are present.