# Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebo

arXiv: https://arxiv.org/abs/1608.05742 · HTML: https://mlsanity.com/paper/1608.05742/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2016.

## Built on by (most cited first, 19 of 19)

- [Gaussian Splatting to Real World Flight Navigation Transfer with Liquid Networks](https://mlsanity.com/paper/2406.15149/) (2024)
- [Shield Synthesis for LTL Modulo Theories](https://mlsanity.com/paper/2406.04184/) (2024)
- [Vision-based Learning for Drones: A Survey](https://mlsanity.com/paper/2312.05019/) (2023)
- [Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular Decomposition](https://mlsanity.com/paper/2110.09018/) (2021)
- [Vision-Based Mobile Robotics Obstacle Avoidance With Deep Reinforcement Learning](https://mlsanity.com/paper/2103.04727/) (2021)
- [MarsExplorer: Exploration of Unknown Terrains via Deep Reinforcement Learning and Procedurally Generated Environments](https://mlsanity.com/paper/2107.09996/) (2021)
- [ASIST: Annotation-free Synthetic Instance Segmentation and Tracking by Adversarial Simulations](https://mlsanity.com/paper/2101.00567/) (2021)
- [ROS-X-Habitat: Bridging the ROS Ecosystem with Embodied AI](https://mlsanity.com/paper/2109.07703/) (2021)
- [Sim-Env: Decoupling OpenAI Gym Environments from Simulation Models](https://mlsanity.com/paper/2102.09824/) (2021)
- [A Scalable and Reproducible System-on-Chip Simulation for Reinforcement Learning](https://mlsanity.com/paper/2104.13187/) (2021)
- [CALF: Communication-Aware Learning Framework for Distributed Reinforcement Learning](https://mlsanity.com/paper/2603.12543/) (2026)
- [A ROS 2 Wrapper for Florence-2: Multi-Mode Local Vision-Language Inference for Robotic Systems](https://mlsanity.com/paper/2604.01179/) (2026)
- [Composable Learning with Sparse Kernel Representations](https://mlsanity.com/paper/2103.14474/) (2021)
- [Reinforcement Learning With Reward Machines in Stochastic Games](https://mlsanity.com/paper/2305.17372/) (2023)
- [Tactile Aware Dynamic Obstacle Avoidance in Crowded Environment with Deep Reinforcement Learning](https://mlsanity.com/paper/2406.13434/) (2024)
- [Autonomous Control of a Novel Closed Chain Five Bar Active Suspension via Deep Reinforcement Learning](https://mlsanity.com/paper/2406.18899/) (2024)
- [RouteFormer: A Transformer-Based Routing Framework for Autonomous Vehicles](https://mlsanity.com/paper/2504.05407/) (2025)
- [Architecture for Multi-Unmanned Aerial Vehicles based Autonomous Precision Agriculture Systems](https://mlsanity.com/paper/2603.21183/) (2026)
- [ARMATA: Auto-Regressive Multi-Agent Task Assignment](https://mlsanity.com/paper/2605.04225/) (2026)

## Abstract (arXiv)

This paper presents an extension of the OpenAI Gym for robotics using the Robot Operating System (ROS) and the Gazebo simulator. The content discusses the software architecture proposed and the results obtained by using two Reinforcement Learning techniques: Q-Learning and Sarsa. Ultimately, the output of this work presents a benchmarking system for robotics that allows different techniques and algorithms to be compared using the same virtual conditions.