# Multi-Timescale Latent-Action DRL for Joint Optimization in Edge-Cloud Networks

arXiv: https://arxiv.org/abs/2607.18288 · HTML: https://mlsanity.com/paper/2607.18288/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 6 of its 40 references that are in the map.
- First posted on arXiv 30 Jun 2026.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 6 of 6)

- [Learning and Planning in Complex Action Spaces](https://mlsanity.com/paper/2104.06303/) (2021)
- [Dynamic Task Offloading and Resource Allocation for Ultra-Reliable Low-Latency Edge Computing](https://mlsanity.com/paper/1812.08076/) (2018)
- [Exact Reduction of Huge Action Spaces in General Reinforcement Learning](https://mlsanity.com/paper/2012.10200/) (2020)
- [Online Resource Procurement and Allocation in a Hybrid Edge-Cloud Computing System](https://mlsanity.com/paper/2001.08956/) (2020)
- [Resilient Edge Service Placement under Demand and Node Failure Uncertainties](https://mlsanity.com/paper/2107.04748/) (2021)
- [Resource Governance in Networked Systems via Integrated Variational Autoencoders and Reinforcement Learning](https://mlsanity.com/paper/2410.23393/) (2024)

## Abstract (arXiv)

Load imbalance across edge and cloud layers degrades latency performance in hierarchical edge-cloud computing (HECC) systems under dynamic task arrivals and heterogeneous resources, leading to severe queuing delays and inefficient resource utilization. To address this challenge, we study a joint service placement, computational delegation, and power control (JSCP) problem to minimize the average end-to-end (e2e) latency. The resulting JSCP problem is a mixed-integer nonconvex and NP-hard optimization problem due to the strong coupling between discrete and continuous variables. To enable tractable optimization and stable system adaptation, we exploit the inherent difference in decision dynamics and decompose the problem into long-term system configuration and short-term resource allocation subproblems. Based on this formulation, we propose a two-timescale multi-layer deep reinforcement learning framework with a latent action space (2T-MDRL-LA) to jointly optimize service placement, user association, computational delegation, task offloading, and user transmit power. A latent action representation based on a variational autoencoder is introduced to efficiently compress the high-dimensional combinatorial action space. Simulation results demonstrate that the proposed framework effectively adapts to dynamic network conditions and achieves near-optimal performance compared to branch-and-bound solutions. It achieves up to a 20.8% reduction in average e2e latency and a 13% improvement in resource utilization over the scheme without the computational delegation, while converging approximately 50% faster than conventional proximal policy optimization.