# Sequential Stochastic Optimization in Separable Learning Environments

arXiv: https://arxiv.org/abs/2108.09585 · HTML: https://mlsanity.com/paper/2108.09585/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 45 references that are in the map.
- First posted on arXiv 21 Aug 2021.

## Builds on (most cited first, 5 of 5)

- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [From Predictive to Prescriptive Analytics](https://mlsanity.com/paper/1402.5481/) (2014)
- [An Approximate Dynamic Programming Algorithm for Monotone Value Functions](https://mlsanity.com/paper/1401.1590/) (2014)
- [From Predictions to Prescriptions in Multistage Optimization Problems](https://mlsanity.com/paper/1904.11637/) (2019)
- [Switching Regression Models and Causal Inference in the Presence of Discrete Latent Variables](https://mlsanity.com/paper/1808.05541/) (2018)

## Abstract (arXiv)

We consider a class of sequential decision-making problems under uncertainty that can encompass various types of supervised learning concepts. These problems have a completely observed state process and a partially observed modulation process, where the state process is affected by the modulation process only through an observation process, the observation process only observes the modulation process, and the modulation process is exogenous to control. We model this broad class of problems as a partially observed Markov decision process (POMDP). The belief function for the modulation process is control invariant, thus separating the estimation of the modulation process from the control of the state process. We call this specially structured POMDP the separable POMDP, or SEP-POMDP, and show it (i) can serve as a model for a broad class of application areas, e.g., inventory control, finance, healthcare systems, (ii) inherits value function and optimal policy structure from a set of completely observed MDPs, (iii) can serve as a bridge between classical models of sequential decision making under uncertainty having fully specified model artifacts and such models that are not fully specified and require the use of predictive methods from statistics and machine learning, and (iv) allows for specialized approximate solution procedures.