# Meta-World+: An Improved, Standardized, RL Benchmark

arXiv: https://arxiv.org/abs/2505.11289 · HTML: https://mlsanity.com/paper/2505.11289/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 16 May 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Contrastive Learning as Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2206.07568/) (2022)
- [ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI](https://mlsanity.com/paper/2410.00425/) (2024)
- [Multi-Task Reinforcement Learning with Context-based Representations](https://mlsanity.com/paper/2102.06177/) (2021)
- [Multi-Task Reinforcement Learning with Soft Modularization](https://mlsanity.com/paper/2003.13661/) (2020)
- [Multi-task Deep Reinforcement Learning with PopArt](https://mlsanity.com/paper/1809.04474/) (2018)
- [A Tutorial on Meta-Reinforcement Learning](https://mlsanity.com/paper/2301.08028/) (2023)
- [XLand-MiniGrid: Scalable Meta-Reinforcement Learning Environments in JAX](https://mlsanity.com/paper/2312.12044/) (2023)
- [Multi-Task Reinforcement Learning with Mixture of Orthogonal Experts](https://mlsanity.com/paper/2311.11385/) (2023)
- [PaCo: Parameter-Compositional Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2210.11653/) (2022)
- [One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotion](https://mlsanity.com/paper/2409.06366/) (2024)
- [Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agents](https://mlsanity.com/paper/2102.02926/) (2021)
- [AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers](https://mlsanity.com/paper/2411.11188/) (2024)
- [Multi-Task Reinforcement Learning Enables Parameter Scaling](https://mlsanity.com/paper/2503.05126/) (2025)
- [Investigating the role of model-based learning in exploration and transfer](https://mlsanity.com/paper/2302.04009/) (2023)

## Built on by (most cited first, 30 of 43)

- [RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Models](https://mlsanity.com/paper/2510.06710/) (2025)
- [π_textttRL: Online RL Fine-tuning for Flow-based Vision-Language-Action Models](https://mlsanity.com/paper/2510.25889/) (2025)
- [One Token Per Frame: Reconsidering Visual Bandwidth in World Models for VLA Policy](https://mlsanity.com/paper/2605.07931/) (2026)
- [ARM-FM: Automated Reward Machines via Foundation Models for Compositional Reinforcement Learning](https://mlsanity.com/paper/2510.14176/) (2025)
- [Stable Deep Reinforcement Learning via Isotropic Gaussian Representations](https://mlsanity.com/paper/2602.19373/) (2026)
- [Drift-Based Policy Optimization: Native One-Step Policy Learning for Online Robot Control](https://mlsanity.com/paper/2604.03540/) (2026)
- [Trajectory Conditioned Cross-embodiment Skill Transfer](https://mlsanity.com/paper/2510.07773/) (2025)
- [Continual Policy Consolidation for Lifelong Robot Learning](https://mlsanity.com/paper/2601.22475/) (2026)
- [ALAM: Algebraically Consistent Latent Action Model for Vision-Language-Action Models](https://mlsanity.com/paper/2605.10819/) (2026)
- [Dual-Granularity Contrastive Reward via Generated Episodic Guidance for Efficient Embodied RL](https://mlsanity.com/paper/2602.12636/) (2026)
- [Ada3Drift: Adaptive Training-Time Drifting for One-Step 3D Visuomotor Robotic Manipulation](https://mlsanity.com/paper/2603.11984/) (2026)
- [Rewarding DINO: Predicting Dense Rewards with Vision Foundation Models](https://mlsanity.com/paper/2603.16978/) (2026)
- [Beyond Action Residuals: Real-World Robot Policy Steering via Bottleneck Latent Reinforcement Learning](https://mlsanity.com/paper/2605.19919/) (2026)
- [VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Models](https://mlsanity.com/paper/2605.20774/) (2026)
- [PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action Models](https://mlsanity.com/paper/2606.22540/) (2026)
- [Gaze on the Prize: Shaping Visual Attention with Return-Guided Contrastive Learning](https://mlsanity.com/paper/2510.08442/) (2025)
- [ForeDiffusion: Foresight-Conditioned Diffusion Policy via Future View Construction for Robot Manipulation](https://mlsanity.com/paper/2601.12925/) (2026)
- [ReGIL: Retrieval-Guided Imitation Learning from a Single Demonstration](https://mlsanity.com/paper/2606.09381/) (2026)
- [Stage-Transition Dense Reward Modeling for Reinforcement Learning](https://mlsanity.com/paper/2606.31377/) (2026)
- [Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learning](https://mlsanity.com/paper/2607.16534/) (2026)
- [DLAM: Distributional Latent Actions with Temporal Constraints](https://mlsanity.com/paper/2607.27138/) (2026)
- [GaussVLA: Geometry-Aware Spatial Reasoning for Vision-Language-Action Model](https://mlsanity.com/paper/2608.24959/) (2026)
- [Quantifying Generalisation in Imitation Learning](https://mlsanity.com/paper/2509.24784/) (2025)
- [Context Representation via Action-Free Transformer encoder-decoder for Meta Reinforcement Learning](https://mlsanity.com/paper/2512.14057/) (2025)
- [Sceniris: A Fast Procedural Scene Generation Framework](https://mlsanity.com/paper/2512.16896/) (2025)
- [Meta-RL with Bayesian Linear Task Models](https://mlsanity.com/paper/2512.20974/) (2025)
- [Recovering Hidden Reward in Diffusion-Based Policies](https://mlsanity.com/paper/2605.00623/) (2026)
- [HeteroGenManip: Generalizable Manipulation For Heterogeneous Object Interactions](https://mlsanity.com/paper/2605.10201/) (2026)
- [TOPPO: Rethinking PPO for Multi-Task Reinforcement Learning with Critic Balancing](https://mlsanity.com/paper/2605.11473/) (2026)
- [Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Sampling](https://mlsanity.com/paper/2605.14350/) (2026)

## Abstract (arXiv)

Meta-World is widely used for evaluating multi-task and meta-reinforcement learning agents, which are challenged to master diverse skills simultaneously. Since its introduction however, there have been numerous undocumented changes which inhibit a fair comparison of algorithms. This work strives to disambiguate these results from the literature, while also leveraging the past versions of Meta-World to provide insights into multi-task and meta-reinforcement learning benchmark design. Through this process we release a new open-source version of Meta-World (https://github.com/Farama-Foundation/Metaworld/) that has full reproducibility of past results, is more technically ergonomic, and gives users more control over the tasks that are included in a task set.