# Should I send this notification? Optimizing push notifications decision making by modeling the future

arXiv: https://arxiv.org/abs/2202.08812 · HTML: https://mlsanity.com/paper/2202.08812/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 17 Feb 2022.

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Top-K Off-Policy Correction for a REINFORCE Recommender System](https://mlsanity.com/paper/1812.02353/) (2018)
- [Deep Reinforcement Learning in Large Discrete Action Spaces](https://mlsanity.com/paper/1512.07679/) (2015)
- [Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems](https://mlsanity.com/paper/1902.05570/) (2019)
- [RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising](https://mlsanity.com/paper/1808.00720/) (2018)
- [Causal Reinforcement Learning using Observational and Interventional Data](https://mlsanity.com/paper/2106.14421/) (2021)
- [Reinforcement Learning to Rank in E-Commerce Search Engine: Formalization, Analysis, and Application](https://mlsanity.com/paper/1803.00710/) (2018)
- [Reinforcement Learning for Slate-based Recommender Systems: A Tractable Decomposition and Practical Methodology](https://mlsanity.com/paper/1905.12767/) (2019)
- [RecSim NG: Toward Principled Uncertainty Modeling for Recommender Ecosystems](https://mlsanity.com/paper/2103.08057/) (2021)
- [Offline Reinforcement Learning for Mobile Notifications](https://mlsanity.com/paper/2202.03867/) (2022)
- [Learning to Rank For Push Notifications Using Pairwise Expected Regret](https://mlsanity.com/paper/2201.07681/) (2022)

## Built on by (most cited first, 6 of 6)

- [Proactive Service Agents: A Unified Decision Framework, Methods, and Evaluation](https://mlsanity.com/paper/2609.03727/) (2026)
- [Personalized Execution Time Optimization for Billion-Scale Scheduled Jobs](https://mlsanity.com/paper/2203.06158/) (2022)
- [Latent Preference Bandits](https://mlsanity.com/paper/2508.05367/) (2025)
- [Generative Sequential Notification Optimization via Multi-Objective Decision Transformers](https://mlsanity.com/paper/2509.02458/) (2025)
- [Sustained Impact of Agentic Personalisation in Marketing: A Longitudinal Case Study](https://mlsanity.com/paper/2604.08621/) (2026)
- [Latent Order Bandits](https://mlsanity.com/paper/2605.07304/) (2026)

## Abstract (arXiv)

Most recommender systems are myopic, that is they optimize based on the immediate response of the user. This may be misaligned with the true objective, such as creating long term user satisfaction. In this work we focus on mobile push notifications, where the long term effects of recommender system decisions can be particularly strong. For example, sending too many or irrelevant notifications may annoy a user and cause them to disable notifications. However, a myopic system will always choose to send a notification since negative effects occur in the future. This is typically mitigated using heuristics. However, heuristics can be hard to reason about or improve, require retuning each time the system is changed, and may be suboptimal. To counter these drawbacks, there is significant interest in recommender systems that optimize directly for long-term value (LTV). Here, we describe a method for maximising LTV by using model-based reinforcement learning (RL) to make decisions about whether to send push notifications. We model the effects of sending a notification on the user's future behavior. Much of the prior work applying RL to maximise LTV in recommender systems has focused on session-based optimization, while the time horizon for notification decision making in this work extends over several days. We test this approach in an A/B test on a major social network. We show that by optimizing decisions about push notifications we are able to send less notifications and obtain a higher open rate than the baseline system, while generating the same level of user engagement on the platform as the existing, heuristic-based, system.