# Double Deep Q-Learning-based Path Selection and Service Placement for Latency-Sensitive Beyond 5G Applications

arXiv: https://arxiv.org/abs/2309.10180 · HTML: https://mlsanity.com/paper/2309.10180/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 48 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Builds on (most cited first, 5 of 5)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [CDN Slicing over a Multi-Domain Edge Cloud](https://mlsanity.com/paper/2201.03999/) (2022)
- [Orchestrating 5G Network Slices to Support Industrial Internet and to Shape Next-Generation Smart Factories](https://mlsanity.com/paper/2201.07185/) (2022)
- [On Multi-domain Network Slicing Orchestration Architecture & Federated Resource Control](https://mlsanity.com/paper/2201.07712/) (2022)

## Built on by (most cited first, 4 of 4)

- [A Semantic-Aware Multiple Access Scheme for Distributed, Dynamic 6G-Based Applications](https://mlsanity.com/paper/2401.06308/) (2024)
- [Towards a Dynamic Future with Adaptable Computing and Network Convergence (ACNC)](https://mlsanity.com/paper/2403.07573/) (2024)
- [ORIENT: A Priority-Aware Energy-Efficient Approach for Latency-Sensitive Applications in 6G](https://mlsanity.com/paper/2402.06931/) (2024)
- [Fairness-Utilization Trade-off in Wireless Networks with Explainable Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2411.01924/) (2024)

## Abstract (arXiv)

Nowadays, as the need for capacity continues to grow, entirely novel services are emerging. A solid cloud-network integrated infrastructure is necessary to supply these services in a real-time responsive, and scalable way. Due to their diverse characteristics and limited capacity, communication and computing resources must be collaboratively managed to unleash their full potential. Although several innovative methods have been proposed to orchestrate the resources, most ignored network resources or relaxed the network as a simple graph, focusing only on cloud resources. This paper fills the gap by studying the joint problem of communication and computing resource allocation, dubbed CCRA, including function placement and assignment, traffic prioritization, and path selection considering capacity constraints and quality requirements, to minimize total cost. We formulate the problem as a non-linear programming model and propose two approaches, dubbed B&B-CCRA and WF-CCRA, based on the Branch & Bound and Water-Filling algorithms to solve it when the system is fully known. Then, for partially known systems, a Double Deep Q-Learning (DDQL) architecture is designed. Numerical simulations show that B&B-CCRA optimally solves the problem, whereas WF-CCRA delivers near-optimal solutions in a substantially shorter time. Furthermore, it is demonstrated that DDQL-CCRA obtains near-optimal solutions in the absence of request-specific information.