MDP modeling for multi-stage stochastic programs
Not on the map. Why?
Abstract (arXiv)
We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include decision-dependent uncertainty for one-step transition probabilities as well as a limited form of statistical learning. We focus on the expressiveness of our modeling approach, illustrating ideas with a series of examples of increasing complexity. As a solution method, we develop new variants of stochastic dual dynamic programming, including approximations to handle non-convexities.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 8 of its 59 references that are in the map. First posted on arXiv 26 Sep 2025.
Builds on
8 of 59 references.
References · 8
- 2015Contextual Markov Decision Processescited by 220
- 2013A General Framework for Updating Belief Distributionscited by 175
- 2021Policy Mirror Descent for Reinforcement Learning: Linear Convergence, New Sampling Complexity, and Generalized Problem Classescited by 156
- 2022JuMP 1.0: Recent improvements to a modeling language for mathematical optimizationcited by 37
- 2018Deep Hierarchical Reinforcement Learning Algorithm in Partially Observable Markov Decision Processescited by 8
- 2019Adaptive Two-stage Stochastic Programming with an Analysis on Capacity Expansion Planning Problemcited by 2
- 2022Risk Filtering and Risk-Averse Control of Markovian Systems Subject to Model Uncertaintycited by 2
- 2014Convergence analysis of sampling-based decomposition methods for risk-averse multistage stochastic convex programscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.