# panda-gym: Open-source goal-conditioned environments for robotic learning

arXiv: https://arxiv.org/abs/2106.13687 · HTML: https://mlsanity.com/paper/2106.13687/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 13 references that are in the map.
- First posted on arXiv 25 Jun 2021.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research](https://mlsanity.com/paper/1802.09464/) (2018)
- [Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1810.10191/) (2018)

## Built on by (most cited first, 30 of 59)

- [Learning Multimodal Behaviors from Scratch with Diffusion Policy Gradient](https://mlsanity.com/paper/2406.00681/) (2024)
- [Code as Reward: Empowering Reinforcement Learning with VLMs](https://mlsanity.com/paper/2402.04764/) (2024)
- [Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning](https://mlsanity.com/paper/2502.10550/) (2025)
- [Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Critic](https://mlsanity.com/paper/2306.02865/) (2023)
- [Temporal Disentanglement of Representations for Improved Generalisation in Reinforcement Learning](https://mlsanity.com/paper/2207.05480/) (2022)
- [Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learning](https://mlsanity.com/paper/2402.03046/) (2024)
- [ACE : Off-Policy Actor-Critic with Causality-Aware Entropy Regularization](https://mlsanity.com/paper/2402.14528/) (2024)
- [A Survey of Imitation Learning Methods, Environments and Metrics](https://mlsanity.com/paper/2404.19456/) (2024)
- [RLingua: Improving Reinforcement Learning Sample Efficiency in Robotic Manipulations With Large Language Models](https://mlsanity.com/paper/2403.06420/) (2024)
- [ContraBAR: Contrastive Bayes-Adaptive Deep RL](https://mlsanity.com/paper/2306.02418/) (2023)
- [Multi-view Disentanglement for Reinforcement Learning with Multiple Cameras](https://mlsanity.com/paper/2404.14064/) (2024)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [Efficient Multi-Task and Transfer Reinforcement Learning with Parameter-Compositional Framework](https://mlsanity.com/paper/2306.01839/) (2023)
- [Understanding when Dynamics-Invariant Data Augmentations Benefit Model-Free Reinforcement Learning Updates](https://mlsanity.com/paper/2310.17786/) (2023)
- [SPRINQL: Sub-optimal Demonstrations driven Offline Imitation Learning](https://mlsanity.com/paper/2402.13147/) (2024)
- [Cell-Free Latent Go-Explore](https://mlsanity.com/paper/2208.14928/) (2022)
- [Offline Goal-Conditioned Reinforcement Learning for Safety-Critical Tasks with Recovery Policy](https://mlsanity.com/paper/2403.01734/) (2024)
- [OMPO: A Unified Framework for RL under Policy and Dynamics Shifts](https://mlsanity.com/paper/2405.19080/) (2024)
- [Precision-Focused Reinforcement Learning Model for Robotic Object Pushing](https://mlsanity.com/paper/2411.08622/) (2024)
- [Mutual Information Tracks Policy Coherence in Reinforcement Learning](https://mlsanity.com/paper/2509.10423/) (2025)
- [Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Robotics](https://mlsanity.com/paper/2204.04308/) (2022)
- [Programmatic Imitation Learning from Unlabeled and Noisy Demonstrations](https://mlsanity.com/paper/2303.01440/) (2023)
- [Logical Specifications-guided Dynamic Task Sampling for Reinforcement Learning Agents](https://mlsanity.com/paper/2402.03678/) (2024)
- [Stealthy Imitation: Reward-guided Environment-free Policy Stealing](https://mlsanity.com/paper/2405.07004/) (2024)
- [Unsupervised Skill Discovery for Robotic Manipulation through Automatic Task Generation](https://mlsanity.com/paper/2410.04855/) (2024)
- [Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challenges](https://mlsanity.com/paper/2508.17449/) (2025)
- [Self-Supervised Multisensory Pretraining for Contact-Rich Robot Reinforcement Learning](https://mlsanity.com/paper/2511.14427/) (2025)
- [GRAIL: Goal Recognition Alignment through Imitation Learning](https://mlsanity.com/paper/2602.14252/) (2026)
- [Automaton-Guided Curriculum Generation for Reinforcement Learning Agents](https://mlsanity.com/paper/2304.05271/) (2023)
- [Optimizing Deep Reinforcement Learning for Adaptive Robotic Arm Control](https://mlsanity.com/paper/2407.02503/) (2024)

## Abstract (arXiv)

This paper presents panda-gym, a set of Reinforcement Learning (RL) environments for the Franka Emika Panda robot integrated with OpenAI Gym. Five tasks are included: reach, push, slide, pick & place and stack. They all follow a Multi-Goal RL framework, allowing to use goal-oriented RL algorithms. To foster open-research, we chose to use the open-source physics engine PyBullet. The implementation chosen for this package allows to define very easily new tasks or new robots. This paper also presents a baseline of results obtained with state-of-the-art model-free off-policy algorithms. panda-gym is open-source and freely available at https://github.com/qgallouedec/panda-gym.