# Efficient iterative policy optimization

arXiv: https://arxiv.org/abs/1612.08967 · HTML: https://mlsanity.com/paper/1612.08967/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Dec 2016.

## Built on by (most cited first, 3 of 3)

- [Tapered Off-Policy REINFORCE: Stable and efficient reinforcement learning for LLMs](https://mlsanity.com/paper/2503.14286/) (2025)
- [Supervised Fine Tuning on Curated Data is Reinforcement Learning (and can be improved)](https://mlsanity.com/paper/2507.12856/) (2025)
- [Boosted Off-Policy Learning](https://mlsanity.com/paper/2208.01148/) (2022)

## Abstract (arXiv)

We tackle the issue of finding a good policy when the number of policy updates is limited. This is done by approximating the expected policy reward as a sequence of concave lower bounds which can be efficiently maximized, drastically reducing the number of policy updates required to achieve good performance. We also extend existing methods to negative rewards, enabling the use of control variates.