# Robust and Scalable Routing with Multi-Agent Deep Reinforcement Learning for MANETs

arXiv: https://arxiv.org/abs/2101.03273 · HTML: https://mlsanity.com/paper/2101.03273/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 40 references that are in the map.
- First posted on arXiv 9 Jan 2021.

## Builds on (most cited first, 7 of 7)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [A Deep-Reinforcement Learning Approach for Software-Defined Networking Routing Optimization](https://mlsanity.com/paper/1709.07080/) (2017)
- [Parameter Sharing Deep Deterministic Policy Gradient for Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1710.00336/) (2017)
- [Hierarchical Deep Double Q-Routing](https://mlsanity.com/paper/1910.04041/) (2019)

## Built on by (most cited first, 5 of 5)

- [Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey](https://mlsanity.com/paper/2110.13484/) (2021)
- [Causal Multi-Agent Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2111.06721/) (2021)
- [Learning an Adaptive Forwarding Strategy for Mobile Wireless Networks: Resource Usage vs. Latency](https://mlsanity.com/paper/2207.11386/) (2022)
- [DeepADMR: A Deep Learning based Anomaly Detection for MANET Routing](https://mlsanity.com/paper/2302.13877/) (2023)
- [Context-Aware Hybrid Routing in Bluetooth Mesh Networks Using Multi-Model Machine Learning and AODV Fallback](https://mlsanity.com/paper/2509.21490/) (2025)

## Abstract (arXiv)

Highly dynamic mobile ad-hoc networks (MANETs) are continuing to serve as one of the most challenging environments to develop and deploy robust, efficient, and scalable routing protocols. In this paper, we present DeepCQ+ routing which, in a novel manner, integrates emerging multi-agent deep reinforcement learning (MADRL) techniques into existing Q-learning-based routing protocols and their variants, and achieves persistently higher performance across a wide range of MANET configurations while training only on a limited range of network parameters and conditions. Quantitatively, DeepCQ+ shows consistently higher end-to-end throughput with lower overhead compared to its Q-learning-based counterparts with the overall gain of 10-15% in its efficiency. Qualitatively and more significantly, DeepCQ+ maintains remarkably similar performance gains under many scenarios that it was not trained for in terms of network sizes, mobility conditions, and traffic dynamics. To the best of our knowledge, this is the first successful demonstration of MADRL for the MANET routing problem that achieves and maintains a high degree of scalability and robustness even in the environments that are outside the trained range of scenarios. This implies that the proposed hybrid design approach of DeepCQ+ that combines MADRL and Q-learning significantly increases its practicality and explainability because the real-world MANET environment will likely vary outside the trained range of MANET scenarios.