# Semantic-Aware Collaborative Deep Reinforcement Learning Over Wireless Cellular Networks

arXiv: https://arxiv.org/abs/2111.12064 · HTML: https://mlsanity.com/paper/2111.12064/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 25 references that are in the map.
- First posted on arXiv 23 Nov 2021.

## Builds on (most cited first, 16 of 16)

- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients](https://mlsanity.com/paper/2010.01264/) (2020)
- [Distral: Robust Multitask Reinforcement Learning](https://mlsanity.com/paper/1707.04175/) (2017)
- [A Study on Overfitting in Deep Reinforcement Learning](https://mlsanity.com/paper/1804.06893/) (2018)
- [A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems](https://mlsanity.com/paper/1902.10265/) (2019)
- [Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning](https://mlsanity.com/paper/1703.02949/) (2017)
- [Federated Reinforcement Learning: Techniques, Applications, and Open Challenges](https://mlsanity.com/paper/2108.11887/) (2021)
- [Dual Policy Distillation](https://mlsanity.com/paper/2006.04061/) (2020)
- [REPAINT: Knowledge Transfer in Deep Reinforcement Learning](https://mlsanity.com/paper/2011.11827/) (2020)
- [A New Representation of Successor Features for Transfer across Dissimilar Environments](https://mlsanity.com/paper/2107.08426/) (2021)
- [Robust Domain Randomised Reinforcement Learning through Peer-to-Peer Distillation](https://mlsanity.com/paper/2012.04839/) (2020)
- [Collaborative Deep Reinforcement Learning for Joint Object Search](https://mlsanity.com/paper/1702.05573/) (2017)
- [Integrated Millimeter Wave and Sub-6 GHz Wireless Networks: A Roadmap for Joint Mobile Broadband and Ultra-Reliable Low-Latency Communications](https://mlsanity.com/paper/1802.03837/) (2018)
- [A Taxonomy of Similarity Metrics for Markov Decision Processes](https://mlsanity.com/paper/2103.04706/) (2021)
- [Collaborative Deep Reinforcement Learning](https://mlsanity.com/paper/1702.05796/) (2017)
- [Multimodal Reward Shaping for Efficient Exploration in Reinforcement Learning](https://mlsanity.com/paper/2107.08888/) (2021)

## Built on by (most cited first, 6 of 6)

- [Curriculum Learning for Goal-Oriented Semantic Communications with a Common Language](https://mlsanity.com/paper/2204.10429/) (2022)
- [Evolutionary Deep Reinforcement Learning for Dynamic Slice Management in O-RAN](https://mlsanity.com/paper/2208.14394/) (2022)
- [Semantics-Empowered Communication: A Tutorial-cum-Survey](https://mlsanity.com/paper/2212.08487/) (2022)
- [Open RAN LSTM Traffic Prediction and Slice Management using Deep Reinforcement Learning](https://mlsanity.com/paper/2401.06922/) (2024)
- [Triplet Loss-less Center Loss Sampling Strategies in Facial Expression Recognition Scenarios](https://mlsanity.com/paper/2302.04108/) (2023)
- [Joint Path planning and Power Allocation of a Cellular-Connected UAV using Apprenticeship Learning via Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/2306.10071/) (2023)

## Abstract (arXiv)

Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach to enable future intelligent and autonomous systems that rely on real-time decision-making in complex dynamic environments. Nonetheless, in practical scenarios, CDRL faces many challenges due to the heterogeneity of agents and their learning tasks, different environments, time constraints of the learning, and resource limitations of wireless networks. To address these challenges, in this paper, a novel semantic-aware CDRL method is proposed to enable a group of heterogeneous untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network. To this end, a new heterogeneous federated DRL (HFDRL) algorithm is proposed to select the best subset of semantically relevant DRL agents for collaboration. The proposed approach then jointly optimizes the training loss and wireless bandwidth allocation for the cooperating selected agents in order to train each agent within the time limit of its real-time task. Simulation results show the superior performance of the proposed algorithm compared to state-of-the-art baselines.