mlsanityData to 7 Oct 2026

GRUtopia: Dream General Robots in a City at Scale

arXiv 2407.10943 ↗First posted 15 Jul 2024cs.RO cs.CVcited by 54

Not on the map. Why?

Abstract (arXiv)

Recent works have been exploring the scaling laws in the field of Embodied AI. Given the prohibitive costs of collecting real-world data, we believe the Simulation-to-Real (Sim2Real) paradigm is a crucial step for scaling the learning of embodied models. This paper introduces project GRUtopia, the first simulated interactive 3D society designed for various robots. It features several advancements: (a) The scene dataset, GRScenes, includes 100k interactive, finely annotated scenes, which can be freely combined into city-scale environments. In contrast to previous works mainly focusing on home, GRScenes covers 89 diverse scene categories, bridging the gap of service-oriented environments where general robots would be initially deployed. (b) GRResidents, a Large Language Model (LLM) driven Non-Player Character (NPC) system that is responsible for social interaction, task generation, and task assignment, thus simulating social scenarios for embodied AI applications. (c) The benchmark, GRBench, supports various robots but focuses on legged robots as primary agents and poses moderately challenging tasks involving Object Loco-Navigation, Social Loco-Navigation, and Loco-Manipulation. We hope that this work can alleviate the scarcity of high-quality data in this field and provide a more comprehensive assessment of Embodied AI research. The project is available at https://github.com/OpenRobotLab/GRUtopia.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 48 of its 57 references that are in the map. First posted on arXiv 15 Jul 2024.

Builds on

48 of 57 references.

References · 48

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2023Qwen Technical Reportcited by 3,273
  3. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  4. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  5. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  6. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  7. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  8. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  9. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  10. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  11. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  12. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
36 more
  1. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  2. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  3. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  4. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  5. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  6. 2018On Evaluation of Embodied Navigation Agentscited by 526
  7. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  8. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  9. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  10. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  11. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  12. 2019Learning agile and dynamic motor skills for legged robotscited by 402
  13. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  14. 2024InternLM2 Technical Reportcited by 391
  15. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  16. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  17. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  18. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  19. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  20. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  21. 2019Vision-and-Dialog Navigationcited by 246
  22. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  23. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
  24. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  25. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  26. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
  27. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 151
  28. 2023GenSim: Generating Robotic Simulation Tasks via Large Language Modelscited by 120
  29. 2020SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulationcited by 118
  30. 2021Visual Room Rearrangementcited by 101
  31. 2021ManipulaTHOR: A Framework for Visual Object Manipulationcited by 101
  32. 20193-D Scene Graph: A Sparse and Semantic Representation of Physical Environments for Intelligent Agentscited by 74
  33. 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 68
  34. 2021The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AIcited by 64
  35. 2024Visual Whole-Body Control for Legged Loco-Manipulationcited by 58
  36. 2023Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Responsecited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 27; 2026: 24.

54 papers.

Sort:

By area:

  1. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  2. 2025Visual Spatial Tuningcited by 85
  3. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 67
  4. 2025HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpitcited by 65
  5. 2025NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidancecited by 46
  6. 2025InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policycited by 46
  7. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environmentcited by 42
  8. 2026Articraft: An Agentic System for Scalable Articulated 3D Asset Generationcited by 25
  9. 20253D Scene Generation: A Surveycited by 24
  10. 2025Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparitiescited by 24
  11. 2024UrbanWorld: An Urban World Model for 3D City Generationcited by 23
  12. 2025Particulate: Feed-Forward 3D Object Articulationcited by 20
38 more
  1. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluationcited by 17
  2. 2025Compositional Generative Model of Unbounded 4D Citiescited by 16
  3. 2025Towards Autonomous Micromobility through Scalable Urban Simulationcited by 16
  4. 2025Virtual Community: An Open World for Humans, Robots, and Societycited by 13
  5. 2026SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learningcited by 13
  6. 2025SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worldscited by 10
  7. 2026TIC-VLA: A Think-in-Control Vision-Language-Action Model for Robot Navigation in Dynamic Environmentscited by 8
  8. 2026MANSION: Multi-floor lANguage-to-3D Scene generatIOn for loNg-horizon taskscited by 8
  9. 2025NavComposer: Composing Language Instructions for Navigation Trajectories through Action-Scene-Object Modularizationcited by 5
  10. 2025MajutsuCity: Language-driven Aesthetic-adaptive City Generation with Controllable 3D Assets and Layoutscited by 5
  11. 2025Mind and Motion Aligned: A Joint Evaluation IsaacSim Benchmark for Task Planning and Low-Level Policies in Mobile Manipulationcited by 4
  12. 2025LoGoPlanner: Localization Grounded Navigation Policy with Metric-aware Visual Geometrycited by 4
  13. 2026Imagine a City: CityGenAgent for Procedural 3D City Generationcited by 4
  14. 2025Agent-as-a-Service based on Agent Networkcited by 3
  15. 2025MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learningcited by 3
  16. 2025TongSIM: A General Platform for Simulating Intelligent Machinescited by 3
  17. 2026See, Remember, Explore: A Benchmark and Baselines for Streaming Spatial Reasoningcited by 3
  18. 2025EMMOE: A Comprehensive Benchmark for Embodied Mobile Manipulation in Open Environmentscited by 2
  19. 2025FLAM: Foundation Model-Based Body Stabilization for Humanoid Locomotion and Manipulationcited by 2
  20. 2025RealMirror: A Comprehensive, Open-Source Vision-Language-Action Platform for Embodied AIcited by 2
  21. 2025UrbanWorld2.0: A Multimodal Agentic Framework for Reality-Aligned 3D World Generation at City-Scalecited by 2
  22. 2025Arcadia: Toward a Full-Lifecycle Framework for Embodied Lifelong Learningcited by 2
  23. 2026STRNet: Visual Navigation with Spatio-Temporal Representation through Dynamic Graph Aggregationcited by 2
  24. 2026Instruct-Particulate: Scaling Feed-Forward 3D Object Articulation with Kinematic Controlcited by 2
  25. 2026Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulationcited by 2
  26. 2025Evaluating Multimodal Large Language Models with Daily Composite Tasks in Home Environmentscited by 1
  27. 2026SIM1: Physics-Aligned Simulator as Zero-Shot Data Scaler in Deformable Worldscited by 1
  28. 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
  29. 2026GuideFetch: A Task Coordination Framework for Concurrent Navigation and Object Retrieval in Assistive Robot Dogscited by 1
  30. 2026DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculumcited by 1
  31. 2025Body Discovery of Embodied AIcited by 0
  32. 2026MobileManiBench: Simplifying Model Verification for Mobile Manipulationcited by 0
  33. 2026AffordSim: A Scalable Data Generator and Benchmark for Affordance-Aware Robotic Manipulationcited by 0
  34. 2026H-OmniStereo: Zero-Shot Omnidirectional Stereo Matching with Heading-Aligned Normal Priorscited by 0
  35. 2026Fisher-Preserving Guidance: Training-Free Manifold Constraints for Safe Diffusion Controlcited by 0
  36. 2026UniETP: Unifying Environments for Generalizable Embodied Task Planningcited by 0
  37. 2026NavVerse: Benchmarking Indoor-to-Outdoor Embodied Navigation in Continuous Robot Simulationcited by 0
  38. 2026EA-Nav: Learning Safe Visual Navigation Policies with Embodiment Awarenesscited by 0

50 most cited of 54 All 54 (JSON)