# Learning to Collaborate in Multi-Module Recommendation via Multi-Agent Reinforcement Learning without Communication

arXiv: https://arxiv.org/abs/2008.09369 · HTML: https://mlsanity.com/paper/2008.09369/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Aug 2020.

## Built on by (most cited first, 8 of 8)

- [Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents](https://mlsanity.com/paper/2402.12327/) (2024)
- [A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions](https://mlsanity.com/paper/2109.03540/) (2021)
- [On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems](https://mlsanity.com/paper/2308.11336/) (2023)
- [Online POI Recommendation: Learning Dynamic Geo-Human Interactions in Streams](https://mlsanity.com/paper/2201.10983/) (2022)
- [DM2: Decentralized Multi-Agent Reinforcement Learning for Distribution Matching](https://mlsanity.com/paper/2206.00233/) (2022)
- [Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges](https://mlsanity.com/paper/2305.10091/) (2023)
- [Multi Agent Path Finding using Evolutionary Game Theory](https://mlsanity.com/paper/2212.02010/) (2022)
- [Model-free Reinforcement Learning with Stochastic Reward Stabilization for Recommender Systems](https://mlsanity.com/paper/2308.13246/) (2023)

## Abstract (arXiv)

With the rise of online e-commerce platforms, more and more customers prefer to shop online. To sell more products, online platforms introduce various modules to recommend items with different properties such as huge discounts. A web page often consists of different independent modules. The ranking policies of these modules are decided by different teams and optimized individually without cooperation, which might result in competition between modules. Thus, the global policy of the whole page could be sub-optimal. In this paper, we propose a novel multi-agent cooperative reinforcement learning approach with the restriction that different modules cannot communicate. Our contributions are three-fold. Firstly, inspired by a solution concept in game theory named correlated equilibrium, we design a signal network to promote cooperation of all modules by generating signals (vectors) for different modules. Secondly, an entropy-regularized version of the signal network is proposed to coordinate agents' exploration of the optimal global policy. Furthermore, experiments based on real-world e-commerce data demonstrate that our algorithm obtains superior performance over baselines.