# Self-Sustaining Multiple Access with Continual Deep Reinforcement Learning for Dynamic Metaverse Applications

arXiv: https://arxiv.org/abs/2309.10177 · HTML: https://mlsanity.com/paper/2309.10177/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 17 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Builds on (most cited first, 8 of 8)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Towards Continual Reinforcement Learning: A Review and Perspectives](https://mlsanity.com/paper/2012.13490/) (2020)
- [A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments](https://mlsanity.com/paper/2005.10619/) (2020)
- [Same State, Different Task: Continual Reinforcement Learning without Interference](https://mlsanity.com/paper/2106.02940/) (2021)
- [Deep-Reinforcement Learning Multiple Access for Heterogeneous Wireless Networks](https://mlsanity.com/paper/1712.00162/) (2017)
- [Non-Uniform Time-Step Deep Q-Network for Carrier-Sense Multiple Access in Heterogeneous Wireless Networks](https://mlsanity.com/paper/1910.05221/) (2019)
- [A Deep Reinforcement Learning Framework for Contention-Based Spectrum Sharing](https://mlsanity.com/paper/2110.02736/) (2021)
- [Deep Reinforcement Learning for Random Access in Machine-Type Communication](https://mlsanity.com/paper/2201.09841/) (2022)

## Built on by (most cited first, 2 of 2)

- [A Semantic-Aware Multiple Access Scheme for Distributed, Dynamic 6G-Based Applications](https://mlsanity.com/paper/2401.06308/) (2024)
- [Fairness-Utilization Trade-off in Wireless Networks with Explainable Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2411.01924/) (2024)

## Abstract (arXiv)

The Metaverse is a new paradigm that aims to create a virtual environment consisting of numerous worlds, each of which will offer a different set of services. To deal with such a dynamic and complex scenario, considering the stringent quality of service requirements aimed at the 6th generation of communication systems (6G), one potential approach is to adopt self-sustaining strategies, which can be realized by employing Adaptive Artificial Intelligence (Adaptive AI) where models are continually re-trained with new data and conditions. One aspect of self-sustainability is the management of multiple access to the frequency spectrum. Although several innovative methods have been proposed to address this challenge, mostly using Deep Reinforcement Learning (DRL), the problem of adapting agents to a non-stationary environment has not yet been precisely addressed. This paper fills in the gap in the current literature by investigating the problem of multiple access in multi-channel environments to maximize the throughput of the intelligent agent when the number of active User Equipments (UEs) may fluctuate over time. To solve the problem, a Double Deep Q-Learning (DDQL) technique empowered by Continual Learning (CL) is proposed to overcome the non-stationary situation, while the environment is unknown. Numerical simulations demonstrate that, compared to other well-known methods, the CL-DDQL algorithm achieves significantly higher throughputs with a considerably shorter convergence time in highly dynamic scenarios.