# Deep RL Dual Sourcing Inventory Management with Supply and Capacity Risk Awareness

arXiv: https://arxiv.org/abs/2507.14446 · HTML: https://mlsanity.com/paper/2507.14446/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 19 Jul 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 15 of 15)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon](https://mlsanity.com/paper/1811.06128/) (2018)
- [Neural Combinatorial Optimization with Reinforcement Learning](https://mlsanity.com/paper/1611.09940/) (2016)
- [Learning Combinatorial Optimization Algorithms over Graphs](https://mlsanity.com/paper/1704.01665/) (2017)
- [Reinforcement Learning for Solving the Vehicle Routing Problem](https://mlsanity.com/paper/1802.04240/) (2018)
- [Hindsight Learning for MDPs with Exogenous Inputs](https://mlsanity.com/paper/2207.06272/) (2022)
- [ORL: Reinforcement Learning Benchmarks for Online Stochastic Optimization Problems](https://mlsanity.com/paper/1911.10641/) (2019)
- [Neural Coordination and Capacity Control for Inventory Management](https://mlsanity.com/paper/2410.02817/) (2024)
- [Learning chordal extensions](https://mlsanity.com/paper/1910.07600/) (2019)
- [Learning an Inventory Control Policy with General Inventory Arrival Dynamics](https://mlsanity.com/paper/2310.17168/) (2023)
- [A machine learning framework for neighbor generation in metaheuristic search](https://mlsanity.com/paper/2212.11451/) (2022)
- [Consensus Planning with Primal, Dual, and Proximal Agents](https://mlsanity.com/paper/2408.16462/) (2024)
- [Asymptotic optimality of Tailored Base-Surge policies in dual-sourcing inventory systems](https://mlsanity.com/paper/1503.01071/) (2015)
- [Single- vs. Dual-Policy Reinforcement Learning for Dynamic Bike Rebalancing](https://mlsanity.com/paper/2402.03589/) (2024)
- [Dual Policy Reinforcement Learning for Real-time Rebalancing in Bike-sharing Systems](https://mlsanity.com/paper/2406.00868/) (2024)

## Built on by (most cited first, 1 of 1)

- [Hard Constraints, Smooth Gradients: Learning Feasible Inventory Policies via Differentiable Projection](https://mlsanity.com/paper/2608.02343/) (2026)

## Abstract (arXiv)

In this work, we study how to efficiently apply reinforcement learning (RL) for solving large-scale stochastic optimization problems by leveraging intervention models. The key of the proposed methodology is to better explore the solution space by simulating and composing the stochastic processes using pre-trained deep learning (DL) models. We demonstrate our approach on a challenging real-world application, the multi-sourcing multi-period inventory management problem in supply chain optimization. In particular, we employ deep RL models for learning and forecasting the stochastic supply chain processes under a range of assumptions. Moreover, we also introduce a constraint coordination mechanism, designed to forecast dual costs given the cross-products constraints in the inventory network. We highlight that instead of directly modeling the complex physical constraints into the RL optimization problem and solving the stochastic problem as a whole, our approach breaks down those supply chain processes into scalable and composable DL modules, leading to improved performance on large real-world datasets. We also outline open problems for future research to further investigate the efficacy of such models.