# PIC4rl-gym: a ROS2 modular framework for Robots Autonomous Navigation with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2211.10714 · HTML: https://mlsanity.com/paper/2211.10714/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 19 references that are in the map.
- First posted on arXiv 19 Nov 2022.

## Builds on (most cited first, 16 of 16)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Robot Operating System 2: Design, Architecture, and Uses In The Wild](https://mlsanity.com/paper/2211.07752/) (2022)
- [Socially Aware Motion Planning with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.08862/) (2017)
- [Autonomous Drone Racing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.08624/) (2021)
- [Benchmarking Metric Ground Navigation](https://mlsanity.com/paper/2008.13315/) (2020)
- [APPL: Adaptive Planner Parameter Learning](https://mlsanity.com/paper/2105.07620/) (2021)
- [Dynamically Feasible Deep Reinforcement Learning Policy for Robot Navigation in Dense Mobile Crowds](https://mlsanity.com/paper/2010.14838/) (2020)
- [APPLR: Adaptive Planner Parameter Learning from Reinforcement](https://mlsanity.com/paper/2011.00397/) (2020)
- [Position-Agnostic Autonomous Navigation in Vineyards with Deep Reinforcement Learning](https://mlsanity.com/paper/2206.14155/) (2022)
- [gym-gazebo2, a toolkit for reinforcement learning using ROS 2 and Gazebo](https://mlsanity.com/paper/1903.06278/) (2019)
- [Marvin: an Innovative Omni-Directional Robotic Assistant for Domestic Environments](https://mlsanity.com/paper/2112.05597/) (2021)
- [RL-DWA Omnidirectional Motion Planning for Person Following in Domestic Assistance and Monitoring](https://mlsanity.com/paper/2211.04993/) (2022)
- [ROS2Learn: a reinforcement learning framework for ROS 2](https://mlsanity.com/paper/1903.06282/) (2019)
- [DeepSim: A Reinforcement Learning Environment Build Toolkit for ROS and Gazebo](https://mlsanity.com/paper/2205.08034/) (2022)

## Built on by (most cited first, 4 of 4)

- [Socially aware navigation for mobile robots: a survey on deep reinforcement learning approaches](https://mlsanity.com/paper/2512.00049/) (2025)
- [Adaptive Social Force Window Planner with Reinforcement Learning](https://mlsanity.com/paper/2404.13678/) (2024)
- [Enhancing Navigation Benchmarking and Perception Data Generation for Row-based Crops in Simulation](https://mlsanity.com/paper/2306.15517/) (2023)
- [GzDRL: Reproducible and Scalable Deep Reinforcement Learning with Gazebo](https://mlsanity.com/paper/2609.13243/) (2026)

## Abstract (arXiv)

Learning agents can optimize standard autonomous navigation improving flexibility, efficiency, and computational cost of the system by adopting a wide variety of approaches. This work introduces the textitPIC4rl-gym, a fundamental modular framework to enhance navigation and learning research by mixing ROS2 and Gazebo, the standard tools of the robotics community, with Deep Reinforcement Learning (DRL). The paper describes the whole structure of the PIC4rl-gym, which fully integrates DRL agent's training and testing in several indoor and outdoor navigation scenarios and tasks. A modular approach is adopted to easily customize the simulation by selecting new platforms, sensors, or models. We demonstrate the potential of our novel gym by benchmarking the resulting policies, trained for different navigation tasks, with a complete set of metrics.