# DeepStock: Reinforcement Learning with Policy Regularizations for Inventory Management

arXiv: https://arxiv.org/abs/2603.19621 · HTML: https://mlsanity.com/paper/2603.19621/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 18 references that are in the map.
- First posted on arXiv 20 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 8 of 8)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [Deep Inventory Management](https://mlsanity.com/paper/2210.03137/) (2022)
- [Structure-Informed Deep Reinforcement Learning for Inventory Management](https://mlsanity.com/paper/2507.22040/) (2025)

## Built on by (most cited first, 5 of 5)

- [InvEvolve: Evolving White-Box Inventory Policies via Large Language Models with Performance Guarantees](https://mlsanity.com/paper/2605.00369/) (2026)
- [OR-Transformer: Scaling Real-Time Decision-Making to 1,000 Items](https://mlsanity.com/paper/2609.01933/) (2026)
- [When to Commit and When to Defer: Maturing Markov Decision Processes under Refining Information and Expiring Opportunities](https://mlsanity.com/paper/2606.18820/) (2026)
- [Hard Constraints, Smooth Gradients: Learning Feasible Inventory Policies via Differentiable Projection](https://mlsanity.com/paper/2608.02343/) (2026)
- [Position: Beyond Training Time, Test-Time Coordination is Essential for Cooperative MARL](https://mlsanity.com/paper/2610.17409/) (2026)

## Abstract (arXiv)

Deep Reinforcement Learning (DRL) provides a general-purpose methodology for training inventory policies that can leverage big data and compute. However, off-the-shelf implementations of DRL have seen mixed success, often plagued by high sensitivity to the hyperparameters used during training. In this paper, we show that by imposing policy regularizations, grounded in classical inventory concepts such as "Base Stock", we can significantly accelerate hyperparameter tuning and improve the final performance of several DRL methods. We report details from a 100% deployment of DRL with policy regularizations on Alibaba's e-commerce platform, Tmall. We also include extensive synthetic experiments, which show that policy regularizations reshape the narrative on what is the best DRL method for inventory management.