# A Meta Reinforcement Learning Approach for Predictive Autoscaling in the Cloud

arXiv: https://arxiv.org/abs/2205.15795 · HTML: https://mlsanity.com/paper/2205.15795/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 31 May 2022.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Learning Continuous Control Policies by Stochastic Value Gradients](https://mlsanity.com/paper/1510.09142/) (2015)
- [Sequential Neural Processes](https://mlsanity.com/paper/1906.10264/) (2019)
- [FIRM: An Intelligent Fine-Grained Resource Management Framework for SLO-Oriented Microservices](https://mlsanity.com/paper/2008.08509/) (2020)
- [Automatic Cloud Resource Scaling Algorithm based on Long Short-Term Memory Recurrent Neural Network](https://mlsanity.com/paper/1701.03295/) (2017)

## Built on by (most cited first, 16 of 16)

- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [Demonstration of DB-GPT: Next Generation Data Interaction System Empowered by Large Language Models](https://mlsanity.com/paper/2404.10209/) (2024)
- [SLOTH: Structured Learning and Task-based Optimization for Time Series Forecasting on Hierarchies](https://mlsanity.com/paper/2302.05650/) (2023)
- [End-to-End Modeling Hierarchical Time Series Using Autoregressive Transformer and Conditional Normalizing Flow based Reconciliation](https://mlsanity.com/paper/2212.13706/) (2022)
- [Continual Learning in Predictive Autoscaling](https://mlsanity.com/paper/2307.15941/) (2023)
- [PEER: Expertizing Domain-Specific Tasks with a Multi-Agent Framework and Tuning Methods](https://mlsanity.com/paper/2407.06985/) (2024)
- [Adaptive GPU Resource Allocation for Multi-Agent Collaborative Reasoning in Serverless Environments](https://mlsanity.com/paper/2512.22149/) (2025)
- [ROMAS: A Role-Based Multi-Agent System for Database monitoring and Planning](https://mlsanity.com/paper/2412.13520/) (2024)
- [Automatic Deduction Path Learning via Reinforcement Learning with Environmental Correction](https://mlsanity.com/paper/2306.10083/) (2023)
- [Deep Optimal Timing Strategies for Time Series](https://mlsanity.com/paper/2310.05479/) (2023)
- [OptScaler: A Collaborative Framework for Robust Autoscaling in the Cloud](https://mlsanity.com/paper/2311.12864/) (2023)
- [Adaptive Two-Stage Cloud Resource Scaling via Hierarchical Multi-Indicator Forecasting and Bayesian Decision-Making](https://mlsanity.com/paper/2408.01000/) (2024)
- [Adaptive Learning on User Segmentation: Universal to Specific Representation via Bipartite Neural Interaction](https://mlsanity.com/paper/2409.14945/) (2024)
- [Multi-Dimensional Autoscaling of Stream Processing Services on Edge Devices](https://mlsanity.com/paper/2510.06882/) (2025)
- [A Heavy-Load-Enhanced and Changeable-Periodicity-Perceived Workload Prediction Network](https://mlsanity.com/paper/2308.01917/) (2023)
- [Leveraging Interpretability in the Transformer to Automate the Proactive Scaling of Cloud Resources](https://mlsanity.com/paper/2409.03103/) (2024)

## Abstract (arXiv)

Predictive autoscaling (autoscaling with workload forecasting) is an important mechanism that supports autonomous adjustment of computing resources in accordance with fluctuating workload demands in the Cloud. In recent works, Reinforcement Learning (RL) has been introduced as a promising approach to learn the resource management policies to guide the scaling actions under the dynamic and uncertain cloud environment. However, RL methods face the following challenges in steering predictive autoscaling, such as lack of accuracy in decision-making, inefficient sampling and significant variability in workload patterns that may cause policies to fail at test time. To this end, we propose an end-to-end predictive meta model-based RL algorithm, aiming to optimally allocate resource to maintain a stable CPU utilization level, which incorporates a specially-designed deep periodic workload prediction model as the input and embeds the Neural Process to guide the learning of the optimal scaling actions over numerous application services in the Cloud. Our algorithm not only ensures the predictability and accuracy of the scaling strategy, but also enables the scaling decisions to adapt to the changing workloads with high sample efficiency. Our method has achieved significant performance improvement compared to the existing algorithms and has been deployed online at Alipay, supporting the autoscaling of applications for the world-leading payment platform.