# Reinforcement Learning-based Product Delivery Frequency Control

arXiv: https://arxiv.org/abs/2012.10858 · HTML: https://mlsanity.com/paper/2012.10858/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2020.

## Built on by (most cited first, 4 of 4)

- [Doubly Robust Distributionally Robust Off-Policy Evaluation and Learning](https://mlsanity.com/paper/2202.09667/) (2022)
- [User Tampering in Reinforcement Learning Recommender Systems](https://mlsanity.com/paper/2109.04083/) (2021)
- [Personalized Execution Time Optimization for Billion-Scale Scheduled Jobs](https://mlsanity.com/paper/2203.06158/) (2022)
- [A hierarchy tree data structure for behavior-based user segment representation](https://mlsanity.com/paper/2508.01115/) (2025)

## Abstract (arXiv)

Frequency control is an important problem in modern recommender systems. It dictates the delivery frequency of recommendations to maintain product quality and efficiency. For example, the frequency of delivering promotional notifications impacts daily metrics as well as the infrastructure resource consumption (e.g. CPU and memory usage). There remain open questions on what objective we should optimize to represent business values in the long term best, and how we should balance between daily metrics and resource consumption in a dynamically fluctuating environment. We propose a personalized methodology for the frequency control problem, which combines long-term value optimization using reinforcement learning (RL) with a robust volume control technique we termed "Effective Factor". We demonstrate statistically significant improvement in daily metrics and resource efficiency by our method in several notification applications at a scale of billions of users. To our best knowledge, our study represents the first deep RL application on the frequency control problem at such an industrial scale.