# Lagrangian Duality in Reinforcement Learning

arXiv: https://arxiv.org/abs/2007.09998 · HTML: https://mlsanity.com/paper/2007.09998/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2020.

## Built on by (most cited first, 1 of 1)

- [A Unified Benchmark for Dynamic Medical Treatment Reinforcement Learning](https://mlsanity.com/paper/2606.01028/) (2026)

## Abstract (arXiv)

Although duality is used extensively in certain fields, such as supervised learning in machine learning, it has been much less explored in others, such as reinforcement learning (RL). In this paper, we show how duality is involved in a variety of RL work, from that which spearheaded the field, such as Richard Bellman's value iteration, to that which was done within just the past few years yet has already had significant impact, such as TRPO, A3C, and GAIL. We show that duality is not uncommon in reinforcement learning, especially when value iteration, or dynamic programming, is used or when first or second order approximations are made to transform initially intractable problems into tractable convex programs.