mlsanityData to 7 Oct 2026

Asynchronous Fractional Multi-Agent Deep Reinforcement Learning for Age-Minimal Mobile Edge Computing

arXiv 2409.16832 ↗First posted 25 Sep 2024cs.LG cs.NIcited by 0

Not on the map. Why?

Abstract (arXiv)

In the realm of emerging real-time networked applications such as cyber-physical systems (CPS), the Age of Information (AoI) has emerged as a pivotal metric for evaluating timeliness. To meet the high computational demands, such as those in smart manufacturing within CPS, mobile edge computing (MEC) presents a promising solution for optimizing computing and reducing AoI. In this work, we study the timeliness of compute-intensive updates and explore jointly optimizing the task updating (when to generate a task) and offloading (where to process a task) policies to minimize AoI. Specifically, we consider edge load dynamics and formulate a task scheduling problem to minimize the expected time-average AoI. Solving this problem is challenging due to the fractional objective introduced by AoI and the asynchronous decision-making of the semi-Markov game (SMG). To this end, we propose a fractional reinforcement learning (RL) framework. We begin by introducing a fractional single-agent RL framework and establish its linear convergence rate. Building on this, we develop a fractional multi-agent RL framework, extend Dinkelbach's method, and demonstrate its equivalence to the inexact Newton's method. Furthermore, we provide the conditions under which the framework achieves linear convergence to the Nash equilibrium (NE). To tackle the challenge of asynchronous decision-making in the SMG, we further design an asynchronous model-free fractional multi-agent RL algorithm, where each mobile device can determine the task updating and offloading decisions without knowing the real-time system dynamics and decisions of other devices. Experimental results show that when compared with the best existing baseline algorithm, our proposed algorithm reduces the average AoI by up to 50.6%.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 84 references that are in the map. First posted on arXiv 25 Sep 2024.

Builds on

33 of 84 references.

References · 33

  1. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  4. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  5. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  6. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  7. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  8. 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
  9. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 180
  10. 2020Age of Information: An Introduction and Surveycited by 64
  11. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  12. 2020Deep Reinforcement Learning for Task Offloading in Mobile Edge Computing Systemscited by 20
21 more
  1. 2022Asynchronous Actor-Critic for Multi-Agent Reinforcement Learningcited by 16
  2. 2019Deep Q-Learning for Nash Equilibria: Nash-DQNcited by 14
  3. 2020Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments using A3C learning and Residual Recurrent Neural Networkscited by 14
  4. 2023A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Servicescited by 14
  5. 2018Survey on Multi-Access Edge Computing for Internet of Things Realizationcited by 12
  6. 2018Sampling for Data Freshness Optimization: Non-linear Age Functionscited by 10
  7. 2021Distributed Reinforcement Learning for Privacy-Preserving Dynamic Edge Cachingcited by 10
  8. 2019Age of Information-Aware Radio Resource Management in Vehicular Networks: A Proactive Deep Reinforcement Learning Perspectivecited by 9
  9. 2021A Reinforcement Learning Approach to Age of Information in Multi-User Networks with HARQcited by 8
  10. 2021Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learningcited by 8
  11. 2021Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learningcited by 7
  12. 2020UAV-to-Device Underlay Communications: Age of Information Minimization by Multi-agent Deep Reinforcement Learningcited by 5
  13. 2021Decentralized Task Offloading in Edge Computing: A Multi-User Multi-Armed Bandit Approachcited by 5
  14. 2019Age-Delay Tradeoffs in Queueing Systemscited by 4
  15. 2020Information Freshness-Aware Task Offloading in Air-Ground Integrated Edge Computing Systemscited by 4
  16. 2019Timely Cloud Computing: Preemption and Waitingcited by 3
  17. 2020Age of Information for Multicast Transmission with Fixed and Random Deadlines in IoT Systemscited by 3
  18. 2023Fractional Deep Reinforcement Learning for Age-Minimal Mobile Edge Computingcited by 3
  19. 2018Age-Minimal Transmission for Energy Harvesting Sensors with Finite Batteries: Online Policiescited by 2
  20. 2020Pricing Fresh Datacited by 2
  21. 2020Vehicular Fog Computing Enabled Real-time Collision Warning via Trajectory Calibrationcited by 1

Built on by

Not cited yet.