# A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1703.04221 · HTML: https://mlsanity.com/paper/1703.04221/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Mar 2017.

## Built on by (most cited first, 16 of 16)

- [On-edge Multi-task Transfer Learning: Model and Practice with Data-driven Task Allocation](https://mlsanity.com/paper/2107.02466/) (2021)
- [Unsupervised Resource Allocation with Graph Neural Networks](https://mlsanity.com/paper/2106.09761/) (2021)
- [CE-NAS: An End-to-End Carbon-Efficient Neural Architecture Search Framework](https://mlsanity.com/paper/2406.01414/) (2024)
- [Deep Reinforcement Learning for Job Scheduling and Resource Management in Cloud Computing: An Algorithm-Level Review](https://mlsanity.com/paper/2501.01007/) (2025)
- [VeRLPy: Python Library for Verification of Digital Designs with Reinforcement Learning](https://mlsanity.com/paper/2108.03978/) (2021)
- [Machine learning for cloud resources management -- An overview](https://mlsanity.com/paper/2101.11984/) (2021)
- [PROMPT: Learning Dynamic Resource Allocation Policies for Network Applications](https://mlsanity.com/paper/2201.07916/) (2022)
- [Machine Learning Empowered Intelligent Data Center Networking: A Survey](https://mlsanity.com/paper/2202.13549/) (2022)
- [Hierarchically Structured Scheduling and Execution of Tasks in a Multi-Agent Environment](https://mlsanity.com/paper/2203.03021/) (2022)
- [The state-of-the-art review on resource allocation problem using artificial intelligence methods on various computing paradigms](https://mlsanity.com/paper/2203.12315/) (2022)
- [RISCLESS: A Reinforcement Learning Strategy to Exploit Unused Cloud Resources](https://mlsanity.com/paper/2205.08350/) (2022)
- [RAPID: Enabling Fast Online Policy Learning in Dynamic Public Cloud Environments](https://mlsanity.com/paper/2304.04797/) (2023)
- [An Advanced Reinforcement Learning Framework for Online Scheduling of Deferrable Workloads in Cloud Computing](https://mlsanity.com/paper/2406.01047/) (2024)
- [SLA-MORL: SLA-Aware Multi-Objective Reinforcement Learning for HPC Resource Optimization](https://mlsanity.com/paper/2508.03509/) (2025)
- [Enhancing Q-Value Updates in Deep Q-Learning via Successor-State Prediction](https://mlsanity.com/paper/2511.03836/) (2025)
- [Revisiting TD Target Aggregation under Uncertainty in Q-Learning](https://mlsanity.com/paper/2608.03069/) (2026)

## Abstract (arXiv)

Automatic decision-making approaches, such as reinforcement learning (RL), have been applied to (partially) solve the resource allocation problem adaptively in the cloud computing system. However, a complete cloud resource allocation framework exhibits high dimensions in state and action spaces, which prohibit the usefulness of traditional RL techniques. In addition, high power consumption has become one of the critical concerns in design and control of cloud computing systems, which degrades system reliability and increases cooling cost. An effective dynamic power management (DPM) policy should minimize power consumption while maintaining performance degradation within an acceptable level. Thus, a joint virtual machine (VM) resource allocation and power management framework is critical to the overall cloud computing system. Moreover, novel solution framework is necessary to address the even higher dimensions in state and action spaces. In this paper, we propose a novel hierarchical framework for solving the overall resource allocation and power management problem in cloud computing systems. The proposed hierarchical framework comprises a global tier for VM resource allocation to the servers and a local tier for distributed power management of local servers. The emerging deep reinforcement learning (DRL) technique, which can deal with complicated control problems with large state space, is adopted to solve the global tier problem. Furthermore, an autoencoder and a novel weight sharing structure are adopted to handle the high-dimensional state space and accelerate the convergence speed. On the other hand, the local tier of distributed server power managements comprises an LSTM based workload predictor and a model-free RL based power manager, operating in a distributed manner.