# An open source Multi-Agent Deep Reinforcement Learning Routing Simulator for satellite networks

arXiv: https://arxiv.org/abs/2407.11047 · HTML: https://mlsanity.com/paper/2407.11047/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 14 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Builds on (most cited first, 7 of 7)

- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Inter-Plane Inter-Satellite Connectivity in Dense LEO Constellations](https://mlsanity.com/paper/2005.07965/) (2020)
- [Continual Deep Reinforcement Learning for Decentralized Satellite Routing](https://mlsanity.com/paper/2405.12308/) (2024)
- [NGSO Constellation Design for Global Connectivity](https://mlsanity.com/paper/2203.16597/) (2022)
- [Exploiting topology awareness for routing in LEO satellite constellations](https://mlsanity.com/paper/2006.12242/) (2020)
- [Multi-Agent Deep Reinforcement Learning for Distributed Satellite Routing](https://mlsanity.com/paper/2402.17666/) (2024)

## Built on by (most cited first, 1 of 1)

- [Putting Data at the Centre of Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2409.12001/) (2024)

## Abstract (arXiv)

This paper introduces an open source simulator for packet routing in Low Earth Orbit Satellite Constellations (LSatCs) considering the dynamic system uncertainties. The simulator, implemented in Python, supports traditional Dijkstra's based routing as well as more advanced learning solutions, specifically Q-Routing and Multi-Agent Deep Reinforcement Learning (MA-DRL) from our previous work. It uses an event-based approach with the SimPy module to accurately simulate packet creation, routing and queuing, providing real-time tracking of queues and latency. The simulator is highly configurable, allowing adjustments in routing policies, traffic, ground and space layer topologies, communication parameters, and learning hyperparameters. Key features include the ability to visualize system motion and track packet paths. Results highlight significant improvements in end-to-end (E2E) latency using Reinforcement Learning (RL)-based routing policies compared to traditional methods. The source code, the documentation and a Jupyter notebook with post-processing results and analysis are available on GitHub.