mlsanityData to 7 Oct 2026

RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots

arXiv 2406.02523 ↗First posted 4 Jun 2024cs.RO cs.AI cs.LGcited by 391

Not on the map. Why?

Cited by 146 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advancements in Artificial Intelligence (AI) have largely been propelled by scaling. In Robotics, scaling is hindered by the lack of access to massive robot datasets. We advocate using realistic physical simulation as a means to scale environments, tasks, and datasets for robot learning methods. We present RoboCasa, a large-scale simulation framework for training generalist robots in everyday environments. RoboCasa features realistic and diverse scenes focusing on kitchen environments. We provide thousands of 3D assets across over 150 object categories and dozens of interactable furniture and appliances. We enrich the realism and diversity of our simulation with generative AI tools, such as object assets from text-to-3D models and environment textures from text-to-image models. We design a set of 100 tasks for systematic evaluation, including composite tasks generated by the guidance of large language models. To facilitate learning, we provide high-quality human demonstrations and integrate automated trajectory generation methods to substantially enlarge our datasets with minimal human burden. Our experiments show a clear scaling trend in using synthetically generated robot data for large-scale imitation learning and show great promise in harnessing simulation data in real-world tasks. Videos and open-source code are available at https://robocasa.ai/

Key facts. Cited by 391 papers in the map (data through 7 Oct 2026). Builds on 41 of its 46 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

41 of 46 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  6. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  7. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  8. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  9. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  10. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  11. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  12. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
29 more
  1. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  2. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  3. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  4. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  5. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  6. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  7. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  8. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  9. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  10. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  11. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  12. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  13. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 208
  14. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  15. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  16. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
  17. 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisitioncited by 156
  18. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 129
  19. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  20. 2023GenSim: Generating Robotic Simulation Tasks via Large Language Modelscited by 120
  21. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrationscited by 114
  22. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperationcited by 103
  23. 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
  24. 2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trialscited by 78
  25. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 68
  26. 2023Imitating Task and Motion Planning with Visuomotor Transformerscited by 63
  27. 2016More than a Million Ways to Be Pushed: A High-Fidelity Experimental Dataset of Planar Pushingcited by 30
  28. 2023Benchmarking Offline Reinforcement Learning on Real-Robot Hardwarecited by 23
  29. 2021A Holistic Approach to Reactive Mobile Manipulationcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 26; 2025: 112; 2026: 253.

391 papers.

Sort:

By area:

  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  2. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  3. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
  4. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  5. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  6. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
  7. 2025World Simulation with Video Foundation Models for Physical AIcited by 183
  8. 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 171
  9. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 168
  10. 2025LIBERO-PRO: Towards Robust and Fair Evaluation of Vision-Language-Action Models Beyond Memorizationcited by 166
  11. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
  12. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 117
38 more
  1. 2025FLARE: Robot Learning with Implicit World Modelingcited by 105
  2. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 104
  3. 2026StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developingcited by 103
  4. 2026A Pragmatic VLA Foundation Modelcited by 97
  5. 2026DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Controlcited by 71
  6. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 68
  7. 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 68
  8. 2025Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generationcited by 65
  9. 2026RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robotscited by 65
  10. 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoisingcited by 65
  11. 2025RoboArena: Distributed Real-World Evaluation of Generalist Robot Policiescited by 63
  12. 2026Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalizationcited by 61
  13. 2025Embodied AI Agents: Modeling the Worldcited by 57
  14. 2025RoboGround: Robotic Manipulation with Grounded Vision-Language Priorscited by 50
  15. 2024GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMscited by 48
  16. 2026InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulationcited by 48
  17. 2026Qwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Modelscited by 47
  18. 2025AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real Worldcited by 46
  19. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  20. 2025InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policycited by 46
  21. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  22. 2024Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learningcited by 44
  23. 2024A Dual Process VLA: Efficient Robotic Manipulation Leveraging VLMcited by 44
  24. 2024STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learningcited by 42
  25. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 42
  26. 2025HAMLET: Switch your Vision-Language-Action Model into a History-Aware Policycited by 42
  27. 2026Learning Latent Action World Models In The Wildcited by 42
  28. 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 42
  29. 2024Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Taskscited by 40
  30. 2026SAGE: Scalable Agentic 3D Scene Generation for Embodied AIcited by 40
  31. 2026RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Designcited by 40
  32. 2025GWM: Towards Scalable Gaussian World Models for Robotic Manipulationcited by 37
  33. 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulationcited by 37
  34. 2025What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?cited by 37
  35. 2024VLM See, Robot Do: Human Demo Video to Robot Action Plan via Vision Language Modelcited by 36
  36. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  37. 2025Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learningcited by 34
  38. 2025ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Contextcited by 34

50 most cited of 391 All 391 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning
  2. 2026RoboJEPA: Scaling Robotic Latent World Models
  3. 2026RoboDream: Compositional World Models for Scalable Robot Data Synthesis
  4. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenes
  5. 2022CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learning
  6. 2026Genie Sim 3.0 : A High-Fidelity Comprehensive Simulation Platform for Humanoid Robot
  7. 2025Real2Render2Real: Scaling Robot Data Without Dynamics Simulation or Robot Hardware
  8. 2023Gen2Sim: Scaling up Robot Learning in Simulation with Generative Models