# Reinforcement Learning When All Actions are Not Always Available

arXiv: https://arxiv.org/abs/1906.01772 · HTML: https://mlsanity.com/paper/1906.01772/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2019.

## Built on by (most cited first, 3 of 3)

- [Algorithmic Censoring in Dynamic Learning Systems](https://mlsanity.com/paper/2305.09035/) (2023)
- [A Bibliometric Analysis and Review on Reinforcement Learning for Transportation Applications](https://mlsanity.com/paper/2210.14524/) (2022)
- [Reinforcement Learning Framework for Server Placement and Workload Allocation in Multi-Access Edge Computing](https://mlsanity.com/paper/2203.07998/) (2022)

## Abstract (arXiv)

The Markov decision process (MDP) formulation used to model many real-world sequential decision making problems does not efficiently capture the setting where the set of available decisions (actions) at each time step is stochastic. Recently, the stochastic action set Markov decision process (SAS-MDP) formulation has been proposed, which better captures the concept of a stochastic action set. In this paper we argue that existing RL algorithms for SAS-MDPs can suffer from potential divergence issues, and present new policy gradient algorithms for SAS-MDPs that incorporate variance reduction techniques unique to this setting, and provide conditions for their convergence. We conclude with experiments that demonstrate the practicality of our approaches on tasks inspired by real-life use cases wherein the action set is stochastic.