mlsanityData to 14 Oct 2026

Deep Inventory Management

arXiv 2210.03137 ↗First posted 6 Oct 2022cs.LG math.OCcited by 23

Not on the map. Why?

Abstract (arXiv)

This work provides a Deep Reinforcement Learning approach to solving a periodic review inventory control system with stochastic vendor lead times, lost sales, correlated demand, and price matching. While this dynamic program has historically been considered intractable, our results show that several policy learning approaches are competitive with or outperform classical methods. In order to train these algorithms, we develop novel techniques to convert historical data into a simulator. On the theoretical side, we present learnability results on a subclass of inventory control problems, where we provide a provable reduction of the reinforcement learning problem to that of supervised learning. On the algorithmic side, we present a model-based reinforcement learning procedure (Direct Backprop) to solve the periodic review inventory control problem by constructing a differentiable simulator. Under a variety of metrics Direct Backprop outperforms model-free RL and newsvendor baselines, in both simulations and real-world deployments.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 18 of its 57 references that are in the map. First posted on arXiv 6 Oct 2022.

Builds on

18 of 57 references.

References · 18

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2016OpenAI Gymcited by 1,736
  6. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  7. 2017A Multi-Horizon Quantile Recurrent Forecastercited by 211
  8. 2018ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Roboticscited by 98
  9. 2022Do Differentiable Simulators Give Better Policy Gradients?cited by 89
  10. 2018Simple random search provides a competitive approach to reinforcement learningcited by 75
  11. 2020MQTransformer: Multi-Horizon Forecasts with Context Dependent and Feedback-Aware Attentioncited by 29
  12. 2022Sample-Efficient Reinforcement Learning in the Presence of Exogenous Informationcited by 26
6 more
  1. 2022Hindsight Learning for MDPs with Exogenous Inputscited by 26
  2. 2016Applying Deep Learning to the Newsvendor Problemcited by 24
  3. 2019ORL: Reinforcement Learning Benchmarks for Online Stochastic Optimization Problemscited by 21
  4. 2019Deep Generative Quantile-Copula Models for Probabilistic Forecastingcited by 18
  5. 2021Sparsity in Partially Controllable Linear Systemscited by 9
  6. 2022MQRetNN: Multi-Horizon Time Series Forecasting with Retrieval Augmentationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 7; 2025: 2; 2026: 8.

23 papers.

Sort:

By area:

  1. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  2. 2023Policy Optimization for Continuous Reinforcement Learningcited by 31
  3. 2023An Analysis of Multi-Agent Reinforcement Learning for Decentralized Inventory Control Systemscited by 11
  4. 2026AI Agents for Inventory Control: Human-LLM-OR Complementaritycited by 8
  5. 2023What model does MuZero learn?cited by 7
  6. 2023Learning an Inventory Control Policy with General Inventory Arrival Dynamicscited by 5
  7. 2025Ask, Clarify, Optimize: Human-LLM Agent Collaboration for Smarter Inventory Controlcited by 5
  8. 2026DeepStock: Reinforcement Learning with Policy Regularizations for Inventory Managementcited by 5
  9. 2024Differentiable Discrete Event Simulation for Queuing Network Controlcited by 4
  10. 2024Exploiting Exogenous Structure for Sample-Efficient Reinforcement Learningcited by 3
  11. 2024The Fallacy of Minimizing Cumulative Regret in the Sequential Task Settingcited by 1
  12. 2024Speeding up Policy Simulation in Supply Chain RLcited by 1
11 more
  1. 2026gym-invmgmt: An Open Benchmarking Framework for Inventory Management Methodscited by 1
  2. 2026Policy Optimization in Hybrid Discrete-Continuous Action Spaces via Mixed Gradientscited by 1
  3. 2026Optimal Hidden-Target Learning for Online Inventory Optimization on General Convex Setscited by 1
  4. 2026OR-Transformer: Scaling Real-Time Decision-Making to 1,000 Itemscited by 1
  5. 2023Contextual Bandits for Evaluating and Improving Inventory Control Policiescited by 0
  6. 2024Pivoting Retail Supply Chain with Deep Generative Techniques: Taxonomy, Survey and Insightscited by 0
  7. 2024Generative Probabilistic Planning for Optimizing Supply Chain Networkscited by 0
  8. 2024Zero-shot Generalization in Inventory Management: Train, then Estimate and Decidecited by 0
  9. 2025The Spoils of Algorithmic Collusion: Profit Allocation Among Asymmetric Firmscited by 0
  10. 2026Is Pure Exploitation Sufficient in Exogenous MDPs with Linear Function Approximation?cited by 0
  11. 2026COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Managementcited by 0