# Energy Efficient Edge Computing: When Lyapunov Meets Distributed Reinforcement Learning

arXiv: https://arxiv.org/abs/2103.16985 · HTML: https://mlsanity.com/paper/2103.16985/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 25 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Survey of Multi-Access Edge Computing in 5G and Beyond: Fundamentals, Technology Integration, and State-of-the-Art](https://mlsanity.com/paper/1906.08452/) (2019)
- [Lyapunov-guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks](https://mlsanity.com/paper/2010.01370/) (2020)
- [Multi-Agent Reinforcement Learning for Adaptive User Association in Dynamic mmWave Networks](https://mlsanity.com/paper/2006.09066/) (2020)
- [A Reinforcement Learning Formulation of the Lyapunov Optimization: Application to Edge Computing Systems with Queue Stability](https://mlsanity.com/paper/2012.07279/) (2020)
- [Energy Efficient Mobile Edge Computing in Dense Cellular Networks](https://mlsanity.com/paper/1701.07405/) (2017)

## Built on by (most cited first, 1 of 1)

- [Lyapunov-Driven Deep Reinforcement Learning for Edge Inference Empowered by Reconfigurable Intelligent Surfaces](https://mlsanity.com/paper/2305.10931/) (2023)

## Abstract (arXiv)

In this work, we study the problem of energy-efficient computation offloading enabled by edge computing. In the considered scenario, multiple users simultaneously compete for limited radio and edge computing resources to get offloaded tasks processed under a delay constraint, with the possibility of exploiting low power sleep modes at all network nodes. The radio resource allocation takes into account inter- and intra-cell interference, and the duty cycles of the radio and computing equipment have to be jointly optimized to minimize the overall energy consumption. To address this issue, we formulate the underlying problem as a dynamic long-term optimization. Then, based on Lyapunov stochastic optimization tools, we decouple the formulated problem into a CPU scheduling problem and a radio resource allocation problem to be solved in a per-slot basis. Whereas the first one can be optimally and efficiently solved using a fast iterative algorithm, the second one is solved using distributed multi-agent reinforcement learning due to its non-convexity and NP-hardness. The resulting framework achieves up to 96.5% performance of the optimal strategy based on exhaustive search, while drastically reducing complexity. The proposed solution also allows to increase the network's energy efficiency compared to a benchmark heuristic approach.