# VC Theory for Inventory Policies

arXiv: https://arxiv.org/abs/2404.11509 · HTML: https://mlsanity.com/paper/2404.11509/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 16 of its 71 references that are in the map.
- First posted on arXiv 17 Apr 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 16 of 16)

- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [Competitive caching with machine learned advice](https://mlsanity.com/paper/1802.05399/) (2018)
- [Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Model](https://mlsanity.com/paper/2005.12900/) (2020)
- [Near-Optimal Offline Reinforcement Learning via Double Variance Reduction](https://mlsanity.com/paper/2102.01748/) (2021)
- [Horizon-Free Reinforcement Learning in Polynomial Time: the Power of Stationary Policies](https://mlsanity.com/paper/2203.12922/) (2022)
- [Hindsight Learning for MDPs with Exogenous Inputs](https://mlsanity.com/paper/2207.06272/) (2022)
- [Learning in structured MDPs with convex cost functions: Improved regret bounds for inventory management](https://mlsanity.com/paper/1905.04337/) (2019)
- [Generalizing Complex Hypotheses on Product Distributions: Auctions, Prophet Inequalities, and Pandora's Problem](https://mlsanity.com/paper/1911.11936/) (2019)
- [Deep Reinforcement Learning for Inventory Networks: Toward Reliable Policy Optimization](https://mlsanity.com/paper/2306.11246/) (2023)
- [Generalization Guarantees for Multi-item Profit Maximization: Pricing, Auctions, and Randomized Mechanisms](https://mlsanity.com/paper/1705.00243/) (2017)
- [How Big Should Your Data Really Be? Data-Driven Newsvendor: Learning One Sample at a Time](https://mlsanity.com/paper/2107.02742/) (2021)
- [Deep Neural Newsvendor](https://mlsanity.com/paper/2309.13830/) (2023)
- [From Contextual Data to Newsvendor Decisions: On the Actual Performance of Data-Driven Algorithms](https://mlsanity.com/paper/2302.08424/) (2023)
- [Optimality gap of constant-order policies decays exponentially in the lead time for lost sales models](https://mlsanity.com/paper/1409.1499/) (2014)
- [Randomized Policy Optimization for Optimal Stopping](https://mlsanity.com/paper/2203.13446/) (2022)
- [Metric Entropy-Free Sample Complexity Bounds for Sample Average Approximation in Convex Stochastic Programming](https://mlsanity.com/paper/2401.00664/) (2024)

## Built on by (most cited first, 8 of 8)

- [Algorithm Configuration for Structured Pfaffian Settings](https://mlsanity.com/paper/2409.04367/) (2024)
- [Neural Coordination and Capacity Control for Inventory Management](https://mlsanity.com/paper/2410.02817/) (2024)
- [Reinforcement Learning with Intrinsically Motivated Feedback Graph for Lost-sales Inventory Control](https://mlsanity.com/paper/2406.18351/) (2024)
- [Survey of Data-driven Newsvendor: Unified Analysis and Spectrum of Achievable Regrets](https://mlsanity.com/paper/2409.03505/) (2024)
- [Policy Optimization in Hybrid Discrete-Continuous Action Spaces via Mixed Gradients](https://mlsanity.com/paper/2605.14297/) (2026)
- [OR-Transformer: Scaling Real-Time Decision-Making to 1,000 Items](https://mlsanity.com/paper/2609.01933/) (2026)
- [A Rigorous, Tractable Measure of Model Complexity](https://mlsanity.com/paper/2605.21167/) (2026)
- [Towards Optimal Inventory Control under Censored Demand: A Biased Sample-Average Approximation Approach](https://mlsanity.com/paper/2609.39397/) (2026)

## Abstract (arXiv)

There has been growing interest in applying reinforcement learning (RL) to inventory management, either by optimizing over temporal transitions or by learning directly from full historical demand trajectories. This contrasts sharply with classical data-driven approaches, which first estimate demand distributions from past data and then compute well-structured optimal policies via dynamic programming. This paper considers a hybrid approach that combines trajectory-based RL with policy regularization imposing base-stock and (s, S)  structures. We provide generalization guarantees for this combined approach for several well-known classes in a T-period dynamic inventory model, using tools from the celebrated Vapnik-Chervonenkis (VC) theory, such as the Pseudo-dimension and Fat-shattering dimension. Our results have implications for regret against the best-in-class policies, and allow for an arbitrary distribution over demand sequences, which makes no assumptions such as independence across time. Surprisingly, we prove that the class of policies defined by T non-stationary base-stock levels exhibits a generalization error that does not grow with T, whereas the two-parameter (s, S) policy class has a generalization error growing logarithmically with T. Overall, our analysis leverages specific inventory structures within the learning theory framework, and improves sample complexity guarantees even compared to existing results assuming independent demands.