# Neural Stochastic Dual Dynamic Programming

arXiv: https://arxiv.org/abs/2112.00874 · HTML: https://mlsanity.com/paper/2112.00874/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 58 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Learning Combinatorial Optimization Algorithms over Graphs](https://mlsanity.com/paper/1704.01665/) (2017)
- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [From Predictive to Prescriptive Analytics](https://mlsanity.com/paper/1402.5481/) (2014)
- [Meta-Q-Learning](https://mlsanity.com/paper/1910.00125/) (2019)
- [OR-Gym: A Reinforcement Learning Library for Operations Research Problems](https://mlsanity.com/paper/2008.06319/) (2020)
- [Applying Deep Learning to the Newsvendor Problem](https://mlsanity.com/paper/1607.02177/) (2016)
- [ORL: Reinforcement Learning Benchmarks for Online Stochastic Optimization Problems](https://mlsanity.com/paper/1911.10641/) (2019)
- [Learning to Plan in High Dimensions via Neural Exploration-Exploitation Trees](https://mlsanity.com/paper/1903.00070/) (2019)
- [Meta Reinforcement Learning with Task Embedding and Shared Policy](https://mlsanity.com/paper/1905.06527/) (2019)
- [VPE: Variational Policy Embedding for Transfer Reinforcement Learning](https://mlsanity.com/paper/1809.03548/) (2018)
- [Scale Up Nonlinear Component Analysis with Doubly Stochastic Gradients](https://mlsanity.com/paper/1504.03655/) (2015)
- [Dynamic Stochastic Approximation for Multi-stage Stochastic Optimization](https://mlsanity.com/paper/1707.03324/) (2017)
- [Complexity of Stochastic Dual Dynamic Programming](https://mlsanity.com/paper/1912.07702/) (2019)
- [Randomized Progressive Hedging methods for Multi-stage Stochastic Programming](https://mlsanity.com/paper/2009.12186/) (2020)

## Built on by (most cited first, 6 of 6)

- [Neur2SP: Neural Two-Stage Stochastic Programming](https://mlsanity.com/paper/2205.12006/) (2022)
- [Fast Continuous and Integer L-shaped Heuristics Through Supervised Learning](https://mlsanity.com/paper/2205.00897/) (2022)
- [Learning to Optimize with Stochastic Dominance Constraints](https://mlsanity.com/paper/2211.07767/) (2022)
- [ICNN-enhanced 2SP: Leveraging input convex neural networks for solving two-stage stochastic programming](https://mlsanity.com/paper/2505.05261/) (2025)
- [Business Metric-Aware Forecasting for Inventory Management](https://mlsanity.com/paper/2308.13118/) (2023)
- [Transformer-based Stagewise Decomposition for Large-Scale Multistage Stochastic Optimization](https://mlsanity.com/paper/2404.02583/) (2024)

## Abstract (arXiv)

Stochastic dual dynamic programming (SDDP) is a state-of-the-art method for solving multi-stage stochastic optimization, widely used for modeling real-world process optimization tasks. Unfortunately, SDDP has a worst-case complexity that scales exponentially in the number of decision variables, which severely limits applicability to only low dimensional problems. To overcome this limitation, we extend SDDP by introducing a trainable neural model that learns to map problem instances to a piece-wise linear value function within intrinsic low-dimension space, which is architected specifically to interact with a base SDDP solver, so that can accelerate optimization performance on new instances. The proposed Neural Stochastic Dual Dynamic Programming (nu-SDDP) continually self-improves by solving successive problems. An empirical investigation demonstrates that nu-SDDP can significantly reduce problem solving cost without sacrificing solution quality over competitors such as SDDP and reinforcement learning algorithms, across a range of synthetic and real-world process optimization problems.