# MO-Playground: Massively Parallelized Multi-Objective Reinforcement Learning for Robotics

arXiv: https://arxiv.org/abs/2603.09237 · HTML: https://mlsanity.com/paper/2603.09237/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 25 of 25)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulation](https://mlsanity.com/paper/2106.13281/) (2021)
- [dm_control: Software and Tasks for Continuous Control](https://mlsanity.com/paper/2006.12983/) (2020)
- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [pymoo: Multi-objective Optimization in Python](https://mlsanity.com/paper/2002.04504/) (2020)
- [BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion](https://mlsanity.com/paper/2508.08241/) (2025)
- [MuJoCo Playground](https://mlsanity.com/paper/2502.08844/) (2025)
- [PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithm](https://mlsanity.com/paper/2208.07914/) (2022)
- [Opt2Skill: Imitating Dynamically-feasible Whole-Body Trajectories for Versatile Humanoid Loco-Manipulation](https://mlsanity.com/paper/2409.20514/) (2024)
- [Preference-Based Learning for Exoskeleton Gait Optimization](https://mlsanity.com/paper/1909.12316/) (2019)
- [ROIAL: Region of Interest Active Learning for Characterizing Exoskeleton Gait Preference Landscapes](https://mlsanity.com/paper/2011.04812/) (2020)
- [Pareto Set Learning for Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2501.06773/) (2025)
- [EvoX: A Distributed GPU-accelerated Framework for Scalable Evolutionary Computation](https://mlsanity.com/paper/2301.12457/) (2023)
- [A First Look at Ad-block Detection: A New Arms Race on the Web](https://mlsanity.com/paper/1605.05841/) (2016)
- [Rating-based Reinforcement Learning](https://mlsanity.com/paper/2307.16348/) (2023)
- [AMOR: Adaptive Character Control through Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2505.23708/) (2025)
- [Learning Pareto Set for Multi-Objective Continuous Robot Control](https://mlsanity.com/paper/2406.18924/) (2024)
- [Studying the Interplay Between the Actor and Critic Representations in Reinforcement Learning](https://mlsanity.com/paper/2503.06343/) (2025)
- [NaviGait: Navigating Dynamically Feasible Gait Libraries using Deep Reinforcement Learning](https://mlsanity.com/paper/2510.11542/) (2025)
- [A Learning-Based Framework for Safe Human-Robot Collaboration with Multiple Backup Control Barrier Functions](https://mlsanity.com/paper/2310.05865/) (2023)

## Built on by (most cited first, 2 of 2)

- [A GPU-Parallel Framework for Heterogeneous Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2606.03335/) (2026)
- [MOCHA: Multi-Objective Co-Design using Hypernetwork Architectures](https://mlsanity.com/paper/2609.30570/) (2026)

## Abstract (arXiv)

Multi-objective reinforcement learning (MORL) is a powerful tool to learn Pareto-optimal policy families across conflicting objectives. However, unlike traditional RL algorithms, existing MORL algorithms do not effectively leverage large-scale parallelization to concurrently simulate thousands of environments, resulting in vastly increased computation time. Ultimately, this has limited MORL's application towards complex multi-objective robotics problems. To address these challenges, we present 1) MORLAX, a new GPU-native, fast MORL algorithm, and 2) MO-Playground, a pip-installable playground of GPU-accelerated multi-objective environments. Together, MORLAX and MO-Playground approximate Pareto sets within minutes, offering 25-270x speed-ups compared to legacy CPU-based approaches whilst achieving superior Pareto front hypervolumes. We demonstrate the versatility of our approach by implementing a custom BRUCE humanoid robot environment using MO-Playground and learning Pareto-optimal locomotion policies across 6 realistic objectives for BRUCE, such as smoothness, efficiency and arm swinging.