# FIRE: A Failure-Adaptive Reinforcement Learning Framework for Edge Computing Migrations

arXiv: https://arxiv.org/abs/2209.14399 · HTML: https://mlsanity.com/paper/2209.14399/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 64 references that are in the map.
- First posted on arXiv 28 Sep 2022.

## Builds on (most cited first, 10 of 10)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Mobile Edge Computing: A Survey on Architecture and Computation Offloading](https://mlsanity.com/paper/1702.05309/) (2017)
- [Efficient Multi-User Computation Offloading for Mobile-Edge Cloud Computing](https://mlsanity.com/paper/1510.00888/) (2015)
- [Optimized Computation Offloading Performance in Virtual Edge Computing Systems via Deep Reinforcement Learning](https://mlsanity.com/paper/1805.06146/) (2018)
- [Deep Learning for Hybrid 5G Services in Mobile Edge Computing Systems: Learn from a Digital Twin](https://mlsanity.com/paper/1907.01523/) (2019)
- [Dynamic Task Offloading and Resource Allocation for Ultra-Reliable Low-Latency Edge Computing](https://mlsanity.com/paper/1812.08076/) (2018)
- [Follow Me at the Edge: Mobility-Aware Dynamic Service Placement for Mobile Edge Computing](https://mlsanity.com/paper/1809.05239/) (2018)
- [Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2003.02157/) (2020)
- [Let's Share VMs: Optimal Placement and Pricing across Base Stations in MEC Systems](https://mlsanity.com/paper/2101.06129/) (2021)
- [Towards Effective Resource Procurement in MEC: a Resource Re-selling Framework](https://mlsanity.com/paper/2203.00825/) (2022)

## Abstract (arXiv)

In edge computing, users' service profiles are migrated due to user mobility. Reinforcement learning (RL) frameworks have been proposed to do so, often trained on simulated data. However, existing RL frameworks overlook occasional server failures, which although rare, impact latency-sensitive applications like autonomous driving and real-time obstacle detection. Nevertheless, these failures (rare events), being not adequately represented in historical training data, pose a challenge for data-driven RL algorithms. As it is impractical to adjust failure frequency in real-world applications for training, we introduce FIRE, a framework that adapts to rare events by training a RL policy in an edge computing digital twin environment. We propose ImRE, an importance sampling-based Q-learning algorithm, which samples rare events proportionally to their impact on the value function. FIRE considers delay, migration, failure, and backup placement costs across individual and shared service profiles. We prove ImRE's boundedness and convergence to optimality. Next, we introduce novel deep Q-learning (ImDQL) and actor critic (ImACRE) versions of our algorithm to enhance scalability. We extend our framework to accommodate users with varying risk tolerances. Through trace driven experiments, we show that FIRE reduces costs compared to vanilla RL and the greedy baseline in the event of failures.