mlsanityData to 14 Oct 2026

ERPPO: Entropy Regularization-based Proximal Policy Optimization

arXiv 2605.13131 ↗First posted 13 May 2026cs.LG cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Multi-Agent Proximal Policy Optimization (MAPPO) is a variant of the Proximal Policy Optimization (PPO) algorithm, specifically tailored for multi-agent reinforcement learning (MARL). MAPPO optimizes cooperative multi-agent settings by employing a centralized critic with decentralized actors. However, in case of multi-dimensional environment, MAPPO can not extract optimal policy due to non-stationary agent observation. To overcome this problem, we introduce a novel approach, Entropy Regularization-based Proximal Policy Optimization (ERPPO). For the policy optimization, we first define the object detection ambiguity under multi-dimensional observation environment. Distributional Spatiotemporal Ambiguity (DSA) learner is trained to estimate object detection uncertainty in non-stationary constraints. Then, we enhance PPO with a novel Entropy Regularization term. This regularization dynamically adjusts the policy update by applying a stronger (L1) regularization in high-ambiguity observation to encourage significant exploratory actions and a weaker (L2) regularization in low-ambiguity observation to stabilize the proximal policy optimization. This approach is designed to enhance the probability of successful object localization in time-critical operations by reducing detection failures and optimizing search policy. Experiments on a testbed with AirSim-based maritime searching scenarios show that the proposed ERPPO improves accuracy performance. Our proposed method improves higher gradient than MAPPO. Qualitative results confirm that ERPPO effectiveness in terms of suppressing false detection in visually uncertain conditions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 20 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

12 of 20 references.

References · 12

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  7. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  8. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  9. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  10. 2019Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learningcited by 93
  11. 2023Detecting Everything in the Open World: Towards Universal Object Detectioncited by 62
  12. 2023Dealing With Non-stationarity in Decentralized Cooperative Multi-Agent Deep Reinforcement Learning via Multi-Timescale Learningcited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025KIPPO: Koopman-Inspired Proximal Policy Optimization
  2. 2025EPO: Entropy-regularized Policy Optimization for LLM Agents Reinforcement Learning
  3. 2026KLip-PPO: A per-sample KL perspective on PPO-Clip
  4. 2024Enhancing Sample Efficiency and Exploration in Reinforcement Learning through the Integration of Diffusion Models and Proximal Policy Optimization
  5. 2024Beyond the Boundaries of Proximal Policy Optimization
  6. 2025Deep Gaussian Process Proximal Policy Optimization
  7. 2026Wasserstein Proximal Policy Gradient
  8. 2025PPO in the Fisher-Rao geometry