mlsanityData to 7 Oct 2026

RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learning

arXiv 2111.02767 ↗First posted 4 Nov 2021cs.LGcited by 31

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

We introduce RLDS (Reinforcement Learning Datasets), an ecosystem for recording, replaying, manipulating, annotating and sharing data in the context of Sequential Decision Making (SDM) including Reinforcement Learning (RL), Learning from Demonstrations, Offline RL or Imitation Learning. RLDS enables not only reproducibility of existing research and easy generation of new datasets, but also accelerates novel research. By providing a standard and lossless format of datasets it enables to quickly test new algorithms on a wider range of tasks. The RLDS ecosystem makes it easy to share datasets without any loss of information and to be agnostic to the underlying original format when applying various data processing pipelines to large collections of datasets. Besides, RLDS provides tools for collecting data generated by either synthetic agents or humans, as well as for inspecting and manipulating the collected data. Ultimately, integration with TFDS facilitates the sharing of RL datasets with the research community.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 51 references that are in the map. First posted on arXiv 4 Nov 2021.

Builds on

31 of 51 references.

Closest ancestors · 4

Same direction, newest first

  1. 2021Offline Reinforcement Learning with Pseudometric Learning2 Mar
  2. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning24 Jun
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  4. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr

Other references · 27

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  7. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  8. 2014OpenML: networked science in machine learningcited by 523
  9. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  10. 2018Behavioral Cloning from Observationcited by 482
  11. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  12. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
15 more
  1. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 203
  2. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  3. 2020The NetHack Learning Environmentcited by 196
  4. 2019Imitation Learning via Off-Policy Distribution Matchingcited by 182
  5. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  6. 2020Primal Wasserstein Imitation Learningcited by 120
  7. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  8. 2020Reinforcement Learning with Videos: Combining Offline Observations with Interactioncited by 80
  9. 2018Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitationcited by 76
  10. 2020An empirical investigation of the challenges of real-world reinforcement learningcited by 75
  11. 2019Making Efficient Use of Demonstrations to Solve Hard Exploration Problemscited by 45
  12. 2020The MAGICAL Benchmark for Robust Imitationcited by 30
  13. 2020Robust Imitation Learning from Noisy Demonstrationscited by 26
  14. 2019Simitate: A Hybrid Imitation Learning Benchmarkcited by 10
  15. 2020Show me the Way: Intrinsic Motivation from Demonstrationscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 2; 2025: 11; 2026: 11.

31 papers.

Sort:

By area:

  1. 2026ABot-M0: VLA Foundation Model for Robotic Manipulation with Action Manifold Learningcited by 94
  2. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  3. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  4. 2023The Generalization Gap in Offline Reinforcement Learningcited by 25
  5. 2025CronusVLA: Towards Efficient and Robust Manipulation via Multi-Frame Vision-Language-Action Modelingcited by 24
  6. 2023Coherent Soft Imitation Learningcited by 21
  7. 2022Dungeons and Data: A Large-Scale NetHack Datasetcited by 16
  8. 2023Advances in Embodied Navigation Using Large Language Models: A Surveycited by 14
  9. 2025MinD: Learning A Dual-System World Model for Real-Time Planning and Implicit Risk Analysiscited by 13
  10. 2025Dexbotic: Open-Source Vision-Language-Action Toolboxcited by 12
  11. 2023ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Designcited by 6
  12. 2025PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attentioncited by 4
19 more
  1. 2026Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignmentcited by 4
  2. 2022Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysiscited by 3
  3. 2023Similarity-based Knowledge Transfer for Cross-Domain Reinforcement Learningcited by 3
  4. 2024RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot Handscited by 3
  5. 2025RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulation Environmentscited by 3
  6. 2025D2E: Scaling Vision-Action Pretraining on Desktop Data for Transfer to Embodied AIcited by 3
  7. 2024Advances in Transformers for Robotic Applications: A Reviewcited by 2
  8. 2025Robo-DM: Data Management For Large Robot Datasetscited by 2
  9. 2025Bridging Embodiment Gaps: Deploying Vision-Language-Action Models on Soft Robotscited by 1
  10. 2026Grounded World Model for Semantically Generalizable Planningcited by 1
  11. 2026RIO: Flexible Real-Time Robot I/O for Cross-Embodiment Robot Learningcited by 1
  12. 2026From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligencecited by 1
  13. 2026FluxVLA Engine: A One-Stop VLA Engineering Platform for Embodied Intelligencecited by 1
  14. 2025Sample-Efficient Robot Skill Learning for Construction Tasks: Benchmarking Hierarchical Reinforcement Learning and Vision-Language-Action VLA Modelcited by 0
  15. 2026ATLAS: An Annotation Tool for Long-horizon Robotic Action Segmentationcited by 0
  16. 2026Vision-Language-Action Models: Experimental Insights from a Real-World UR5 Platformcited by 0
  17. 2026JoyNexus: Service-Oriented Multi-Tenant Post-Training for VLA Modelscited by 0
  18. 2026The Embodiment Gap in Robot Foundation Modelscited by 0
  19. 2026ChunkVLA-AM: Parallel Action Chunking for Vision-Language-Action Robot Control in Additive Manufacturingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025PyTupli: A Scalable Infrastructure for Collaborative Offline Reinforcement Learning Projects
  2. 2020Offline Reinforcement Learning Hands-On
  3. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learning
  4. 2021Conservative Data Sharing for Multi-Task Offline Reinforcement Learning
  5. 2023Using Offline Data to Speed Up Reinforcement Learning in Procedurally Generated Environments
  6. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning
  7. 2024D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning
  8. 2021Representation Matters: Offline Pretraining for Sequential Decision Making