# RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learning

arXiv: https://arxiv.org/abs/2111.02767 · HTML: https://mlsanity.com/paper/2111.02767/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 31 of its 51 references that are in the map.
- First posted on arXiv 4 Nov 2021.

## Placed in

- Offline RL and imitation learning from data / [Offline RL algorithms](https://mlsanity.com/offlinerl/offline-rl-algorithms/)

## Builds on (most cited first, 30 of 31)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [Transporter Networks: Rearranging the Visual World for Robotic Manipulation](https://mlsanity.com/paper/2010.14406/) (2020)
- [Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards](https://mlsanity.com/paper/1707.08817/) (2017)
- [Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning](https://mlsanity.com/paper/1809.02925/) (2018)
- [RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitation](https://mlsanity.com/paper/1811.02790/) (2018)
- [The NetHack Learning Environment](https://mlsanity.com/paper/2006.13760/) (2020)
- [Imitation Learning via Off-Policy Distribution Matching](https://mlsanity.com/paper/1912.05032/) (2019)
- [Acme: A Research Framework for Distributed Reinforcement Learning](https://mlsanity.com/paper/2006.00979/) (2020)
- [Primal Wasserstein Imitation Learning](https://mlsanity.com/paper/2006.04678/) (2020)
- [Benchmarks for Deep Off-Policy Evaluation](https://mlsanity.com/paper/2103.16596/) (2021)
- [Reinforcement Learning with Videos: Combining Offline Observations with Interaction](https://mlsanity.com/paper/2011.06507/) (2020)
- [An empirical investigation of the challenges of real-world reinforcement learning](https://mlsanity.com/paper/2003.11881/) (2020)
- [Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation](https://mlsanity.com/paper/1810.07121/) (2018)
- [RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.13888/) (2020)
- [Making Efficient Use of Demonstrations to Solve Hard Exploration Problems](https://mlsanity.com/paper/1909.01387/) (2019)
- [The MAGICAL Benchmark for Robust Imitation](https://mlsanity.com/paper/2011.00401/) (2020)
- [Offline Reinforcement Learning with Pseudometric Learning](https://mlsanity.com/paper/2103.01948/) (2021)
- [Robust Imitation Learning from Noisy Demonstrations](https://mlsanity.com/paper/2010.10181/) (2020)
- [Simitate: A Hybrid Imitation Learning Benchmark](https://mlsanity.com/paper/1905.06002/) (2019)

## Built on by (most cited first, 30 of 31)

- [ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learning](https://mlsanity.com/paper/2602.11236/) (2026)
- [Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications](https://mlsanity.com/paper/2510.07077/) (2025)
- [Multi-agent Embodied AI: Advances and Future Directions](https://mlsanity.com/paper/2505.05108/) (2025)
- [The Generalization Gap in Offline Reinforcement Learning](https://mlsanity.com/paper/2312.05742/) (2023)
- [CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modeling](https://mlsanity.com/paper/2506.19816/) (2025)
- [Coherent Soft Imitation Learning](https://mlsanity.com/paper/2305.16498/) (2023)
- [Dungeons and Data: A Large-Scale NetHack Dataset](https://mlsanity.com/paper/2211.00539/) (2022)
- [Advances in Embodied Navigation Using Large Language Models: A Survey](https://mlsanity.com/paper/2311.00530/) (2023)
- [MinD: Learning A Dual-System World Model for Real-Time Planning and Implicit Risk Analysis](https://mlsanity.com/paper/2506.18897/) (2025)
- [Dexbotic: Open-Source Vision-Language-Action Toolbox](https://mlsanity.com/paper/2510.23511/) (2025)
- [ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Design](https://mlsanity.com/paper/2306.08888/) (2023)
- [Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment](https://mlsanity.com/paper/2607.13429/) (2026)
- [Similarity-based Knowledge Transfer for Cross-Domain Reinforcement Learning](https://mlsanity.com/paper/2312.03764/) (2023)
- [RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot Hands](https://mlsanity.com/paper/2408.11048/) (2024)
- [D2E: Scaling Vision-Action Pretraining on Desktop Data for Transfer to Embodied AI](https://mlsanity.com/paper/2510.05684/) (2025)
- [PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attention](https://mlsanity.com/paper/2512.03724/) (2025)
- [Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysis](https://mlsanity.com/paper/2206.09046/) (2022)
- [RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environments](https://mlsanity.com/paper/2509.17057/) (2025)
- [Advances in Transformers for Robotic Applications: A Review](https://mlsanity.com/paper/2412.10599/) (2024)
- [Robo-DM: Data Management For Large Robot Datasets](https://mlsanity.com/paper/2505.15558/) (2025)
- [Bridging Embodiment Gaps: Deploying Vision-Language-Action Models on Soft Robots](https://mlsanity.com/paper/2510.17369/) (2025)
- [Grounded World Model for Semantically Generalizable Planning](https://mlsanity.com/paper/2604.11751/) (2026)
- [RIO: Flexible Real-Time Robot I/O for Cross-Embodiment Robot Learning](https://mlsanity.com/paper/2605.11564/) (2026)
- [From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence](https://mlsanity.com/paper/2607.11689/) (2026)
- [The Embodiment Gap in Robot Foundation Models](https://mlsanity.com/paper/2608.18433/) (2026)
- [FluxVLA Engine: A One-Stop VLA Engineering Platform for Embodied Intelligence](https://mlsanity.com/paper/2609.17210/) (2026)
- [Sample-Efficient Robot Skill Learning for Construction Tasks: Benchmarking Hierarchical Reinforcement Learning and Vision-Language-Action VLA Model](https://mlsanity.com/paper/2512.14031/) (2025)
- [ATLAS: An Annotation Tool for Long-horizon Robotic Action Segmentation](https://mlsanity.com/paper/2604.26637/) (2026)
- [Vision-Language-Action Models: Experimental Insights from a Real-World UR5 Platform](https://mlsanity.com/paper/2606.30456/) (2026)
- [JoyNexus: Service-Oriented Multi-Tenant Post-Training for VLA Models](https://mlsanity.com/paper/2607.16074/) (2026)

## Abstract (arXiv)

We introduce RLDS (Reinforcement Learning Datasets), an ecosystem for recording, replaying, manipulating, annotating and sharing data in the context of Sequential Decision Making (SDM) including Reinforcement Learning (RL), Learning from Demonstrations, Offline RL or Imitation Learning. RLDS enables not only reproducibility of existing research and easy generation of new datasets, but also accelerates novel research. By providing a standard and lossless format of datasets it enables to quickly test new algorithms on a wider range of tasks. The RLDS ecosystem makes it easy to share datasets without any loss of information and to be agnostic to the underlying original format when applying various data processing pipelines to large collections of datasets. Besides, RLDS provides tools for collecting data generated by either synthetic agents or humans, as well as for inspecting and manipulating the collected data. Ultimately, integration with TFDS facilitates the sharing of RL datasets with the research community.