mlsanityData to 14 Oct 2026

Policy gradient methods for ordinal policies

arXiv 2506.18614 ↗First posted 23 Jun 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

In reinforcement learning, the softmax parametrization is the standard approach for policies over discrete action spaces. However, it fails to capture the order relationship between actions. Motivated by a real-world industrial problem, we propose a novel policy parametrization based on ordinal regression models adapted to the reinforcement learning setting. Our approach addresses practical challenges, and numerical experiments demonstrate its effectiveness in real applications and in continuous action tasks, where discretizing the action space and applying the ordinal policy yields competitive performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 2 of its 5 references that are in the map. First posted on arXiv 23 Jun 2025.

Builds on

2 of 5 references.

References · 2

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Residual Policy Gradient: A Reward View of KL-regularized Objective
  2. 2023Reparameterized Policy Learning for Multimodal Trajectory Optimization
  3. 2026Ranking-aware Reinforcement Learning for Ordinal Ranking
  4. 2026Policy Optimization in Hybrid Discrete-Continuous Action Spaces via Mixed Gradients
  5. 2023Stepsize Learning for Policy Gradient Methods in Contextual Markov Decision Processes
  6. 2023The In-Sample Softmax for Offline Reinforcement Learning
  7. 2024Matrix Low-Rank Approximation For Policy Gradient Methods
  8. 2025Context-Sensitive Abstractions for Reinforcement Learning with Parameterized Actions