mlsanityData to 14 Oct 2026

MaRCA: Multi-Agent Reinforcement Learning for Dynamic Computation Allocation in Large-Scale Recommender Systems

arXiv 2512.24325 ↗First posted 30 Dec 2025cs.IR cs.LG cs.MAcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern recommender systems face significant computational challenges due to growing model complexity and traffic scale, making efficient computation allocation critical for maximizing business revenue. Existing approaches typically simplify multi-stage computation resource allocation, neglecting inter-stage dependencies, thus limiting global optimality. In this paper, we propose MaRCA, a multi-agent reinforcement learning framework for end-to-end computation resource allocation in large-scale recommender systems. MaRCA models the stages of a recommender system as cooperative agents, using Centralized Training with Decentralized Execution (CTDE) to optimize revenue under computation resource constraints. We introduce an AutoBucket TestBench for accurate computation cost estimation, and a Model Predictive Control (MPC)-based Revenue-Cost Balancer to proactively forecast traffic loads and adjust the revenue-cost trade-off accordingly. Since its end-to-end deployment in the advertising pipeline of a leading global e-commerce platform in November 2024, MaRCA has consistently handled hundreds of billions of ad requests per day and has delivered a 16.67% revenue uplift using existing computation resources.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 46 references that are in the map. First posted on arXiv 30 Dec 2025.

Builds on

24 of 46 references.

References · 24

  1. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  3. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  4. 2016Wide & Deep Learning for Recommender Systemscited by 747
  5. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  6. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  7. 2017Deep & Cross Network for Ad Click Predictionscited by 393
  8. 2020Weighted QMIX: Expanding Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 197
  9. 2016Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learningcited by 106
  10. 2018Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertisingcited by 65
  11. 2021Unified Conversational Recommendation Policy Learning via Graph-based Reinforcement Learningcited by 51
  12. 2019DEAR: Deep Reinforcement Learning for Online Advertising Impression in Recommender Systemscited by 44
12 more
  1. 2020Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learningcited by 42
  2. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 33
  3. 2017Cascade Ranking for Operational E-commerce Searchcited by 29
  4. 2020Jointly Learning to Recommend and Advertisecited by 27
  5. 2025Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Surveycited by 14
  6. 2021Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectivenesscited by 12
  7. 2023COPR: Consistency-Oriented Pre-Ranking for Online Advertisingcited by 10
  8. 2020DCAF: A Dynamic Computation Allocation Framework for Online Serving Systemcited by 7
  9. 2024UNEX-RL: Reinforcing Long-Term Rewards in Multi-Stage Recommender Systems with UNidirectional EXecutioncited by 5
  10. 2021Computation Resource Allocation Solution in Recommender Systemscited by 3
  11. 2023TwERC: High Performance Ensembled Candidate Generation for Ads Recommendation at Twittercited by 2
  12. 2024Cooperative Multi-Agent Deep Reinforcement Learning in Content Ranking Optimizationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking Recommendation Paradigms: From Pipelines to Agentic Recommender Systems
  2. 2024Multi-Scenario Combination Based on Multi-Agent Reinforcement Learning to Optimize the Advertising Recommendation System
  3. 2024Cache-Aware Reinforcement Learning in Large-Scale Recommender Systems
  4. 2026Self-Distilled Reinforcement Learning for Co-Evolving Agentic Recommender Systems
  5. 2025Closing the Loop: Coordinating Inventory and Recommendation via Deep Reinforcement Learning on Multiple Timescales
  6. 2026RecRM-Bench: Benchmarking Multidimensional Reward Modeling for Agentic Recommender Systems
  7. 2026EvoRec: Self Evolving Agentic Recommender Systems
  8. 2026Multi-Agent Video Recommenders: Evolution, Patterns, and Open Challenges