# Multi-Agent Reinforcement Learning for Long-Term Network Resource Allocation through Auction: a V2X Application

arXiv: https://arxiv.org/abs/2208.04237 · HTML: https://mlsanity.com/paper/2208.04237/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 91 references that are in the map.
- First posted on arXiv 29 Jul 2022.

## Builds on (most cited first, 20 of 20)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [Large-Scale Study of Curiosity-Driven Learning](https://mlsanity.com/paper/1808.04355/) (2018)
- [A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning](https://mlsanity.com/paper/1711.00832/) (2017)
- [Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1802.05438/) (2018)
- [Training Very Deep Networks](https://mlsanity.com/paper/1507.06228/) (2015)
- [RUDDER: Return Decomposition for Delayed Rewards](https://mlsanity.com/paper/1806.07857/) (2018)
- [Self-Driving Cars: A Survey](https://mlsanity.com/paper/1901.04407/) (2019)
- [Evolution-Guided Policy Gradient in Reinforcement Learning](https://mlsanity.com/paper/1805.07917/) (2018)
- [Optimizing Agent Behavior over Long Time Scales by Transporting Value](https://mlsanity.com/paper/1810.06721/) (2018)
- [Multi-Agent Reinforcement Learning Based Resource Allocation for UAV Networks](https://mlsanity.com/paper/1810.10408/) (2018)
- [Mobile Edge Computing: A Survey on Architecture and Computation Offloading](https://mlsanity.com/paper/1702.05309/) (2017)
- [Efficient Multi-User Computation Offloading for Mobile-Edge Cloud Computing](https://mlsanity.com/paper/1510.00888/) (2015)
- [Decentralized Computation Offloading Game For Mobile Cloud Computing](https://mlsanity.com/paper/1404.3200/) (2014)
- [Optimal Bidding in Repeated Wireless Spectrum Auctions with Budget Constraints](https://mlsanity.com/paper/1608.07357/) (2016)
- [Multi-Agent Distributed Reinforcement Learning for Making Decentralized Offloading Decisions](https://mlsanity.com/paper/2204.02267/) (2022)
- [Learning to Bid Long-Term: Multi-Agent Reinforcement Learning with Long-Term and Sparse Reward in Repeated Auction Games](https://mlsanity.com/paper/2204.02268/) (2022)
- [Learning from Delayed Outcomes via Proxies with Applications to Recommender Systems](https://mlsanity.com/paper/1807.09387/) (2018)

## Abstract (arXiv)

We formulate offloading of computational tasks from a dynamic group of mobile agents (e.g., cars) as decentralized decision making among autonomous agents. We design an interaction mechanism that incentivizes such agents to align private and system goals by balancing between competition and cooperation. In the static case, the mechanism provably has Nash equilibria with optimal resource allocation. In a dynamic environment, this mechanism's requirement of complete information is impossible to achieve. For such environments, we propose a novel multi-agent online learning algorithm that learns with partial, delayed and noisy state information, thus greatly reducing information need. Our algorithm is also capable of learning from long-term and sparse reward signals with varying delay. Empirical results from the simulation of a V2X application confirm that through learning, agents with the learning algorithm significantly improve both system and individual performance, reducing up to 30% of offloading failure rate, communication overhead and load variation, increasing computation resource utilization and fairness. Results also confirm the algorithm's good convergence and generalization property in different environments.