# ns3-gym: Extending OpenAI Gym for Networking Research

arXiv: https://arxiv.org/abs/1810.03943 · HTML: https://mlsanity.com/paper/1810.03943/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2018.

## Built on by (most cited first, 6 of 6)

- [PowerGym: A Reinforcement Learning Environment for Volt-Var Control in Power Distribution Systems](https://mlsanity.com/paper/2109.03970/) (2021)
- [Untangling Braids with Multi-agent Q-Learning](https://mlsanity.com/paper/2109.14502/) (2021)
- [Machine Learning & Wi-Fi: Unveiling the Path Towards AI/ML-Native IEEE 802.11 Networks](https://mlsanity.com/paper/2405.11504/) (2024)
- [GymD2D: A Device-to-Device Underlay Cellular Offload Evaluation Platform](https://mlsanity.com/paper/2101.11188/) (2021)
- [On Enhancing Network Throughput using Reinforcement Learning in Sliced Testbeds](https://mlsanity.com/paper/2412.16673/) (2024)
- [Dual-Graph Multi-Agent Reinforcement Learning for Handover Optimization](https://mlsanity.com/paper/2603.24634/) (2026)

## Abstract (arXiv)

OpenAI Gym is a toolkit for reinforcement learning (RL) research. It includes a large number of well-known problems that expose a common interface allowing to directly compare the performance results of different RL algorithms. Since many years, the ns-3 network simulation tool is the de-facto standard for academic and industry research into networking protocols and communications technology. Numerous scientific papers were written reporting results obtained using ns-3, and hundreds of models and modules were written and contributed to the ns-3 code base. Today as a major trend in network research we see the use of machine learning tools like RL. What is missing is the integration of a RL framework like OpenAI Gym into the network simulator ns-3. This paper presents the ns3-gym framework. First, we discuss design decisions that went into the software. Second, two illustrative examples implemented using ns3-gym are presented. Our software package is provided to the community as open source under a GPL license and hence can be easily extended.