# A Short Note on Soft-max and Policy Gradients in Bandits Problems

arXiv: https://arxiv.org/abs/2007.10297 · HTML: https://mlsanity.com/paper/2007.10297/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2020.

## Built on by (most cited first, 1 of 1)

- [A Lyapunov Analysis of Softmax Policy Gradient for Stochastic Bandits](https://mlsanity.com/paper/2603.26547/) (2026)

## Abstract (arXiv)

This is a short communication on a Lyapunov function argument for softmax in bandit problems. There are a number of excellent papers coming out using differential equations for policy gradient algorithms in reinforcement learning citeagarwal2019optimality,bhandari2019global,mei2020global. We give a short argument that gives a regret bound for the soft-max ordinary differential equation for bandit problems. We derive a similar result for a different policy gradient algorithm, again for bandit problems. For this second algorithm, it is possible to prove regret bounds in the stochastic case citeDW20. At the end, we summarize some ideas and issues on deriving stochastic regret bounds for policy gradients.