# Learning to Remember: Attentive Reinforcement Learning for Edge Serverless Autoscaling

arXiv: https://arxiv.org/abs/2603.28790 · HTML: https://mlsanity.com/paper/2603.28790/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 47 references that are in the map.
- First posted on arXiv 21 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [A Survey and Critique of Multiagent Deep Reinforcement Learning](https://mlsanity.com/paper/1810.05587/) (2018)
- [KIS-S: A GPU-Aware Kubernetes Inference Simulator with RL-Based Auto-Scaling](https://mlsanity.com/paper/2507.07932/) (2025)
- [TEDGE-Caching: Transformer-based Edge Caching Towards 6G Networks](https://mlsanity.com/paper/2112.00633/) (2021)
- [DeepScaler: Holistic Autoscaling for Microservices Based on Spatiotemporal GNN with Adaptive Graph Learning](https://mlsanity.com/paper/2309.00859/) (2023)
- [Cold Start Latency in Serverless Computing: A Systematic Review, Taxonomy, and Future Directions](https://mlsanity.com/paper/2310.08437/) (2023)
- [A Deep Recurrent-Reinforcement Learning Method for Intelligent AutoScaling of Serverless Functions](https://mlsanity.com/paper/2308.05937/) (2023)
- [An SLO Driven and Cost-Aware Autoscaling Framework for Kubernetes](https://mlsanity.com/paper/2512.23415/) (2025)
- [Self Attention with Temporal Prior: Can We Learn More from Arrow of Time?](https://mlsanity.com/paper/2310.18932/) (2023)
- [A Hybrid Reactive-Proactive Auto-scaling Algorithm for SLA-Constrained Edge Computing](https://mlsanity.com/paper/2512.14290/) (2025)

## Abstract (arXiv)

In edge computing, the stochastic and bursty nature of serverless workloads challenges autonomous resource orchestration. Traditional reactive controllers, such as the Kubernetes Horizontal Pod Autoscaler (HPA), suffer from reaction latency, leading to Service Level Objective (SLO) violations during traffic spikes and resource flapping during ramp-downs. While Deep Reinforcement Learning (DRL) offers a pathway toward proactive management, standard agents suffer from textittemporal blindness, an inability to exploit the recent temporal context in non-Markovian edge environments. To bridge this gap, we propose a stability-aware autoscaling framework unifying short-horizon temporal context and control via an Attention-Enhanced Double-Stacked LSTM architecture integrated within a Proximal Policy Optimization (PPO) agent. Unlike shallow recurrent models, our approach employs a learned attention mechanism that weights recent historical states non-uniformly, suppressing high-frequency jitter while preserving the trend that precedes demand shifts. We validate the framework on two independent Kubernetes clusters using real-world Azure Functions traces. Against the single-layer LSTM ablation and the static HPA baseline, our approach reduces P90 latency by ≈67%, and holds average latency within the 50ms hard SLO for 98.8% of the run against 49.6% and 43.5% respectively. Against Kubernetes Event-Driven Autoscaling (KEDA), it matches latency performance at 75% fewer replica-steps and 59% less churn, with P90 hard-SLO violation bursts of at most 5 consecutive intervals against up to 24 for KEDA. These results indicate that mitigating temporal blindness through deep attentive memory improves the reliability and stability of Kubernetes autoscaling under bursty edge workloads.