# Optimistic Proximal Policy Optimization

arXiv: https://arxiv.org/abs/1906.11075 · HTML: https://mlsanity.com/paper/1906.11075/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jun 2019.

## Built on by (most cited first, 3 of 3)

- [Proximal Policy Optimization with Adaptive Threshold for Symmetric Relative Density Ratio](https://mlsanity.com/paper/2203.09809/) (2022)
- [Optimistic Multi-Agent Policy Gradient](https://mlsanity.com/paper/2311.01953/) (2023)
- [How to Enable Uncertainty Estimation in Proximal Policy Optimization](https://mlsanity.com/paper/2210.03649/) (2022)

## Abstract (arXiv)

Reinforcement Learning, a machine learning framework for training an autonomous agent based on rewards, has shown outstanding results in various domains. However, it is known that learning a good policy is difficult in a domain where rewards are rare. We propose a method, optimistic proximal policy optimization (OPPO) to alleviate this difficulty. OPPO considers the uncertainty of the estimated total return and optimistically evaluates the policy based on that amount. We show that OPPO outperforms the existing methods in a tabular task.