# Multi-Armed Bandit for Energy-Efficient and Delay-Sensitive Edge Computing in Dynamic Networks with Uncertainty

arXiv: https://arxiv.org/abs/1904.06258 · HTML: https://mlsanity.com/paper/1904.06258/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2019.

## Built on by (most cited first, 8 of 8)

- [Distributed Task Management in Fog Computing: A Socially Concave Bandit Game](https://mlsanity.com/paper/2203.14572/) (2022)
- [Non-stationary Delayed Combinatorial Semi-Bandit with Causally Related Rewards](https://mlsanity.com/paper/2307.09093/) (2023)
- [Decentralized Task Offloading and Load-Balancing for Mobile Edge Computing in Dense Networks](https://mlsanity.com/paper/2407.00080/) (2024)
- [A Survey on Reinforcement Learning-Aided Caching in Mobile Edge Networks](https://mlsanity.com/paper/2105.05564/) (2021)
- [A Bandit Approach to Online Pricing for Heterogeneous Edge Resource Allocation](https://mlsanity.com/paper/2302.06953/) (2023)
- [Online Learning with Costly Features in Non-stationary Environments](https://mlsanity.com/paper/2307.09388/) (2023)
- [Budgeted Recommendation with Delayed Feedback](https://mlsanity.com/paper/2405.11417/) (2024)
- [Distributed Multi-Agent Bandits Over ErdHos-R'enyi Random Networks](https://mlsanity.com/paper/2510.22811/) (2025)

## Abstract (arXiv)

In the edge computing paradigm, mobile devices offload the computational tasks to an edge server by routing the required data over the wireless network. The full potential of edge computing becomes realized only if a smart device selects the most appropriate server in terms of the latency and energy consumption, among many available ones. The server selection problem is challenging due to the randomness of the environment and lack of prior information about the environment. Therefore, a smart device, which sequentially chooses a server under uncertainty, aims to improve its decision based on the historical time and energy consumption. The problem becomes more complicated in a dynamic environment, where key variables might undergo abrupt changes. To deal with the aforementioned problem, we first analyze the required time and energy to data transmission and processing. We then use the analysis to cast the problem as a budget-limited multi-armed bandit problem, where each arm is associated with a reward and cost, with time-variant statistical characteristics. We propose a policy to solve the formulated problem and prove a regret bound. The numerical results demonstrate the superiority of the proposed method compared to a number of existing solutions.