# Learning to Order for Inventory Systems with Lost Sales and Uncertain Supplies

arXiv: https://arxiv.org/abs/2207.04550 · HTML: https://mlsanity.com/paper/2207.04550/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 62 references that are in the map.
- First posted on arXiv 10 Jul 2022.

## Builds on (most cited first, 4 of 4)

- [Is Q-learning Provably Efficient?](https://mlsanity.com/paper/1807.03765/) (2018)
- [Learning in structured MDPs with convex cost functions: Improved regret bounds for inventory management](https://mlsanity.com/paper/1905.04337/) (2019)
- [Provably Efficient Reinforcement Learning with Aggregated States](https://mlsanity.com/paper/1912.06366/) (2019)
- [Optimality gap of constant-order policies decays exponentially in the lead time for lost sales models](https://mlsanity.com/paper/1409.1499/) (2014)

## Built on by (most cited first, 6 of 6)

- [Augmenting Offline Reinforcement Learning with State-only Interactions](https://mlsanity.com/paper/2402.00807/) (2024)
- [Reinforcement Learning with Intrinsically Motivated Feedback Graph for Lost-sales Inventory Control](https://mlsanity.com/paper/2406.18351/) (2024)
- [A Minibatch-SGD-Based Learning Meta-Policy for Inventory Systems with Myopic Optimal Policy](https://mlsanity.com/paper/2408.16181/) (2024)
- [Zero-shot Generalization in Inventory Management: Train, then Estimate and Decide](https://mlsanity.com/paper/2411.00515/) (2024)
- [Deep Learning for Perishable Inventory Systems with Human Knowledge](https://mlsanity.com/paper/2601.15589/) (2026)
- [Towards Optimal Inventory Control under Censored Demand: A Biased Sample-Average Approximation Approach](https://mlsanity.com/paper/2609.39397/) (2026)

## Abstract (arXiv)

We consider a stochastic lost-sales inventory control system with a lead time L over a planning horizon T. Supply is uncertain, and is a function of the order quantity (due to random yield/capacity, etc). We aim to minimize the T-period cost, a problem that is known to be computationally intractable even under known distributions of demand and supply. In this paper, we assume that both the demand and supply distributions are unknown and develop a computationally efficient online learning algorithm. We show that our algorithm achieves a regret (i.e. the performance gap between the cost of our algorithm and that of an optimal policy over T periods) of O(L+√T) when L≥log(T). We do so by 1) showing our algorithm cost is higher by at most O(L+√T) for any L≥ 0 compared to an optimal constant-order policy under complete information (a well-known and widely-used algorithm) and 2) leveraging its known performance guarantee from the existing literature. To the best of our knowledge, a finite-sample O(√T) (and polynomial in L) regret bound when benchmarked against an optimal policy is not known before in the online inventory control literature. A key challenge in this learning problem is that both demand and supply data can be censored; hence only truncated values are observable. We circumvent this challenge by showing that the data generated under an order quantity q2 allows us to simulate the performance of not only q2 but also q1 for all q1<q2, a key observation to obtain sufficient information even under data censoring. By establishing a high probability coupling argument, we are able to evaluate and compare the performance of different order policies at their steady state within a finite time horizon. Since the problem lacks convexity, we develop an active elimination method that adaptively rules out suboptimal solutions.