# Data-driven inventory management for new products: An adjusted Dyna-Q approach with transfer learning

arXiv: https://arxiv.org/abs/2501.08109 · HTML: https://mlsanity.com/paper/2501.08109/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 5 of its 24 references that are in the map.
- First posted on arXiv 14 Jan 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 5 of 5)

- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees](https://mlsanity.com/paper/1807.03858/) (2018)
- [Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control](https://mlsanity.com/paper/1811.01848/) (2018)
- [Target Transfer Q-Learning and Its Convergence Analysis](https://mlsanity.com/paper/1809.08923/) (2018)
- [Offline Meta-level Model-based Reinforcement Learning Approach for Cold-Start Recommendation](https://mlsanity.com/paper/2012.02476/) (2020)

## Abstract (arXiv)

In this paper, we propose a novel reinforcement learning algorithm for inventory management of newly launched products with no historical demand information. The algorithm follows the classic Dyna-Q structure, balancing the model-free and model-based approaches, while accelerating the training process of Dyna-Q and mitigating the model discrepancy generated by the model-based feedback. Based on the idea of transfer learning, warm-start information from the demand data of existing similar products can be incorporated into the algorithm to further stabilize the early-stage training and reduce the variance of the estimated optimal policy. Our approach is validated through a case study of bakery inventory management with real data. The adjusted Dyna-Q shows up to a 23.7% reduction in average daily cost compared with Q-learning, and up to a 77.5% reduction in training time within the same horizon compared with classic Dyna-Q. By using transfer learning, it can be found that the adjusted Dyna-Q has the lowest total cost, lowest variance in total cost, and relatively low shortage percentages among all the benchmarking algorithms under a 30-day testing.