mlsanityData to 14 Oct 2026

Convex Is Back: Solving Belief MDPs With Convexity-Informed Deep Reinforcement Learning

arXiv 2502.09298 ↗First posted 13 Feb 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We present a novel method for Deep Reinforcement Learning (DRL), incorporating the convex property of the value function over the belief space in Partially Observable Markov Decision Processes (POMDPs). We introduce hard- and soft-enforced convexity as two different approaches, and compare their performance against standard DRL on two well-known POMDP environments, namely the Tiger and FieldVisionRockSample problems. Our findings show that including the convexity feature can substantially increase performance of the agents, as well as increase robustness over the hyperparameter space, especially when testing on out-of-distribution domains. The source code for this work can be found at https://github.com/Dakout/Convex_DRL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 15 of its 37 references that are in the map. First posted on arXiv 13 Feb 2025.

Builds on

15 of 37 references.

References · 15

  1. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  4. 2019On the Convergence of Adam and Beyondcited by 841
  5. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  6. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
  7. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  8. 2016Input Convex Neural Networkscited by 414
  9. 2015Conditional Random Fields as Recurrent Neural Networkscited by 214
  10. 2017Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activationscited by 104
  11. 2015Structured Prediction Energy Networkscited by 59
  12. 2018Managing engineering systems with large state and action spaces through deep reinforcement learningcited by 32
3 more
  1. 2020The Curious Case of Convex Neural Networkscited by 16
  2. 2023POMDP inference and robust solution via deep reinforcement learning: An application to railway optimal maintenancecited by 13
  3. 2023An investigation of belief-free DRL and MCTS for inspection and maintenance planningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Scalable Policy-Based RL Algorithms for POMDPs
  2. 2024Efficient Exploration in Deep Reinforcement Learning: A Novel Bayesian Actor-Critic Algorithm
  3. 2026Mathematical methods of reinforcement learning
  4. 2024Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learning
  5. 2026Convex-Concave Reinforcement Learning
  6. 2024Deep Reinforcement Learning: A Convex Optimization Approach
  7. 2026Finite-State Controllers for (Hidden-Model) POMDPs using Deep Reinforcement Learning
  8. 2025Towards Optimal Adversarial Robust Reinforcement Learning with Infinity Measurement Error