mlsanityData to 7 Oct 2026

Clipped Action Policy Gradient

arXiv 1802.07564 ↗First posted 21 Feb 2018cs.LG cs.AI stat.MLcited by 17

Not on the map. Why?

Abstract (arXiv)

Many continuous control tasks have bounded action spaces. When policy gradient methods are applied to such tasks, out-of-bound actions need to be clipped before execution, while policies are usually optimized as if the actions are not clipped. We propose a policy gradient estimator that exploits the knowledge of actions being clipped to reduce the variance in estimation. We prove that our estimator, named clipped action policy gradient (CAPG), is unbiased and achieves lower variance than the conventional estimator that ignores action bounds. Experimental results demonstrate that CAPG generally outperforms the conventional estimator, indicating that it is a better policy gradient estimator for continuous control tasks. The source code is available at https://github.com/pfnet-research/capg.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 6; 2024: 1; 2025: 1; 2026: 1.

17 papers.

Sort:

  1. 2023KnowGPT: Knowledge Graph based Prompting for Large Language Modelscited by 30
  2. 2021Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimizationcited by 18
  3. 2022Pseudo-Labeled Auto-Curriculum Learning for Semi-Supervised Keypoint Localizationcited by 14
  4. 2023Benchmarking Actor-Critic Deep Reinforcement Learning Algorithms for Robotics Control with Action Constraintscited by 10
  5. 2021Measuring Progress in Deep Reinforcement Learning Sample Efficiencycited by 8
  6. 2021Factored Policy Gradients: Leveraging Structure for Efficient Learning in MOMDPscited by 8
  7. 2022A Risk-Sensitive Approach to Policy Optimizationcited by 6
  8. 2023Policy Gradient Algorithms Implicitly Optimize by Continuationcited by 4
  9. 2021Explaining Off-Policy Actor-Critic From A Bias-Variance Perspectivecited by 2
  10. 2022Remember and Forget Experience Replay for Multi-Agent Reinforcement Learningcited by 2
  11. 2023Clipped-Objective Policy Gradients for Pessimistic Policy Optimizationcited by 2
  12. 2022Provably Efficient Convergence of Primal-Dual Actor-Critic with Nonlinear Function Approximationcited by 1
5 more
  1. 2023Distillation Policy Optimizationcited by 1
  2. 2023Handling Cost and Constraints with Off-Policy Deep Reinforcement Learningcited by 0
  3. 2024Off-OAB: Off-Policy Policy Gradient Method with Optimal Action-Dependent Baselinecited by 0
  4. 2025Navigating Demand Uncertainty in Container Shipping: Deep Reinforcement Learning for Enabling Adaptive and Feasible Master Stowage Planningcited by 0
  5. 2026Where Entropy Is Measured Matters: Policy Geometry in Bounded Continuous-Control PPOcited by 0