Computing optimal policies for managing inventories with noisy observations
On the map: Deep reinforcement learning methods → RL for energy, networks and operations
Abstract (arXiv)
This paper implements the Deep Deterministic Policy Gradient (DDPG) algorithm for computing optimal policies for partially observable single-product periodic review inventory control problems with setup costs and backorders. The decision maker does not know the exact inventory level, but can obtain noise-corrupted observations of them. The goal is to maximize the expected total discounted costs incurred over a finite planning horizon. We also investigate the Gaussian version of this problem with normally distributed initial inventories, demands, and observation noise. We show that expected posterior observations of inventory levels, also called mean beliefs, provide sufficient statistics for the Gaussian problem. Moreover, they can be represented in the form of a Markov Decision Processes for an inventory control system with time-dependent holding costs and demands. Thus, for a Gaussian problem, the there exist (s_t,S_t)-optimal policies based on mean beliefs, and this fact explains the structure of the approximately optimal policies computed by DDPG. For the Gaussian case, we also numerically compare the performance of policies derived from DDPG to optimal policies for discretized versions of the original continuous problem.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 5 of its 24 references that are in the map. First posted on arXiv 26 Jul 2025.
Builds on
5 of 24 references.
References · 5
- 2014Partially Observable Total-Cost Markov Decision Processes with Weakly Continuous Transition Probabilitiescited by 6
- 2021Markov Decision Processes with Incomplete Information and Semi-Uniform Feller Transition Probabilitiescited by 2
- 2015On the Convergence of Optimal Actions for Markov Decision Processes and the Optimality of (s,S) Inventory Policiescited by 1
- 2016Structure of Optimal Solutions to Periodic-Review Total-Cost Inventory Control Models with Convex Costs and Backorders for all Values of Discount Factorscited by 1
- 2023Continuity of Filters for Discrete-Time Control Problems Defined by Explicit Equationscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024VC Theory for Inventory Policies
- 2025Structure-Informed Deep Reinforcement Learning for Inventory Management
- 2025Classical and Deep Reinforcement Learning Inventory Control Policies for Pharmaceutical Supply Chains with Perishability and Non-Stationarity
- 2025Offline Dynamic Inventory and Pricing Strategy: Addressing Censored and Dependent Demand
- 2026DeepStock: Reinforcement Learning with Policy Regularizations for Inventory Management
- 2024Zero-shot Generalization in Inventory Management: Train, then Estimate and Decide
- 2025Reinforcement Learning in MDPs with Information-Ordered Policies
- 2024Neural Coordination and Capacity Control for Inventory Management