# Open-Source Reinforcement Learning Environments Implemented in MuJoCo with Franka Manipulator

arXiv: https://arxiv.org/abs/2312.13788 · HTML: https://mlsanity.com/paper/2312.13788/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2023.

## Built on by (most cited first, 2 of 2)

- [MIMIC-MJX: Neuromechanical Emulation of Animal Behavior](https://mlsanity.com/paper/2511.20532/) (2025)
- [Benchmarking Action Spaces in Reinforcement Learning for Vision-based Robotic Manipulation](https://mlsanity.com/paper/2606.18594/) (2026)

## Abstract (arXiv)

This paper presents three open-source reinforcement learning environments developed on the MuJoCo physics engine with the Franka Emika Panda arm in MuJoCo Menagerie. Three representative tasks, push, slide, and pick-and-place, are implemented through the Gymnasium Robotics API, which inherits from the core of Gymnasium. Both the sparse binary and dense rewards are supported, and the observation space contains the keys of desired and achieved goals to follow the Multi-Goal Reinforcement Learning framework. Three different off-policy algorithms are used to validate the simulation attributes to ensure the fidelity of all tasks, and benchmark results are also given. Each environment and task are defined in a clean way, and the main parameters for modifying the environment are preserved to reflect the main difference. The repository, including all environments, is available at https://github.com/zichunxx/panda_mujoco_gym.