# RAPID: Enabling Fast Online Policy Learning in Dynamic Public Cloud Environments

arXiv: https://arxiv.org/abs/2304.04797 · HTML: https://mlsanity.com/paper/2304.04797/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 81 references that are in the map.
- First posted on arXiv 10 Apr 2023.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Why gradient clipping accelerates training: A theoretical justification for adaptivity](https://mlsanity.com/paper/1905.11881/) (2019)
- [AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients](https://mlsanity.com/paper/2010.07468/) (2020)
- [Generalization and Regularization in DQN](https://mlsanity.com/paper/1810.00123/) (2018)
- [Action Branching Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1711.08946/) (2017)
- [FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices](https://mlsanity.com/paper/2008.08509/) (2020)
- [COSCO: Container Orchestration using Co-Simulation and Gradient Based Optimization for Fog Computing Environments](https://mlsanity.com/paper/2104.14392/) (2021)
- [A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning](https://mlsanity.com/paper/1703.04221/) (2017)
- [Improving Spark Application Throughput Via Memory Aware Task Co-location: A Mixture of Experts Approach](https://mlsanity.com/paper/1710.00610/) (2017)
- [GOSH: Task Scheduling Using Deep Surrogate Models in Fog Computing Environments](https://mlsanity.com/paper/2112.08916/) (2021)
- [PROMPT: Learning Dynamic Resource Allocation Policies for Network Applications](https://mlsanity.com/paper/2201.07916/) (2022)

## Abstract (arXiv)

Resource sharing between multiple workloads has become a prominent practice among cloud service providers, motivated by demand for improved resource utilization and reduced cost of ownership. Effective resource sharing, however, remains an open challenge due to the adverse effects that resource contention can have on high-priority, user-facing workloads with strict Quality of Service (QoS) requirements. Although recent approaches have demonstrated promising results, those works remain largely impractical in public cloud environments since workloads are not known in advance and may only run for a brief period, thus prohibiting offline learning and significantly hindering online learning. In this paper, we propose RAPID, a novel framework for fast, fully-online resource allocation policy learning in highly dynamic operating environments. RAPID leverages lightweight QoS predictions, enabled by domain-knowledge-inspired techniques for sample efficiency and bias reduction, to decouple control from conventional feedback sources and guide policy learning at a rate orders of magnitude faster than prior work. Evaluation on a real-world server platform with representative cloud workloads confirms that RAPID can learn stable resource allocation policies in minutes, as compared with hours in prior state-of-the-art, while improving QoS by 9.0x and increasing best-effort workload performance by 19-43%.