# Edge-Compatible Reinforcement Learning for Recommendations

arXiv: https://arxiv.org/abs/2112.05812 · HTML: https://mlsanity.com/paper/2112.05812/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 19 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Builds on (most cited first, 11 of 11)

- [Top-K Off-Policy Correction for a REINFORCE Recommender System](https://mlsanity.com/paper/1812.02353/) (2018)
- [Off-policy evaluation for slate recommendation](https://mlsanity.com/paper/1605.04812/) (2016)
- [Deep Reinforcement Learning for Page-wise Recommendations](https://mlsanity.com/paper/1805.02343/) (2018)
- [Federated Reinforcement Learning: Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2108.11887/) (2021)
- [Learning a Deep Listwise Context Model for Ranking Refinement](https://mlsanity.com/paper/1804.05936/) (2018)
- [Seq2Slate: Re-ranking and Slate Optimization with RNNs](https://mlsanity.com/paper/1810.02019/) (2018)
- [Beyond Greedy Ranking: Slate Optimization via List-CVAE](https://mlsanity.com/paper/1803.01682/) (2018)
- [Reinforcement Learning based Recommender System using Biclustering Technique](https://mlsanity.com/paper/1801.05532/) (2018)
- [Reinforcement Learning for Strategic Recommendations](https://mlsanity.com/paper/2009.07346/) (2020)
- [Algorithms for slate bandits with non-separable reward functions](https://mlsanity.com/paper/2004.09957/) (2020)
- [Asynchronous Coagent Networks](https://mlsanity.com/paper/1902.05650/) (2019)

## Abstract (arXiv)

Most reinforcement learning (RL) recommendation systems designed for edge computing must either synchronize during recommendation selection or depend on an unprincipled patchwork collection of algorithms. In this work, we build on asynchronous coagent policy gradient algorithms citepkostas2020asynchronous to propose a principled solution to this problem. The class of algorithms that we propose can be distributed over the internet and run asynchronously and in real-time. When a given edge fails to respond to a request for data with sufficient speed, this is not a problem; the algorithm is designed to function and learn in the edge setting, and network issues are part of this setting. The result is a principled, theoretically grounded RL algorithm designed to be distributed in and learn in this asynchronous environment. In this work, we describe this algorithm and a proposed class of architectures in detail, and demonstrate that they work well in practice in the asynchronous setting, even as the network quality degrades.