mlsanityData to 7 Oct 2026

Automated Creation of Digital Cousins for Robust Policy Learning

arXiv 2410.07408 ↗First posted 9 Oct 2024cs.ROcited by 88

Not on the map. Why?

Abstract (arXiv)

Training robot policies in the real world can be unsafe, costly, and difficult to scale. Simulation serves as an inexpensive and potentially limitless source of training data, but suffers from the semantics and physics disparity between simulated and real-world environments. These discrepancies can be minimized by training in digital twins, which serve as virtual replicas of a real scene but are expensive to generate and cannot produce cross-domain generalization. To address these limitations, we propose the concept of digital cousins, a virtual asset or scene that, unlike a digital twin, does not explicitly model a real-world counterpart but still exhibits similar geometric and semantic affordances. As a result, digital cousins simultaneously reduce the cost of generating an analogous virtual environment while also facilitating better robustness during sim-to-real domain transfer by providing a distribution of similar training scenes. Leveraging digital cousins, we introduce a novel method for their automated creation, and propose a fully automated real-to-sim-to-real pipeline for generating fully interactive scenes and training robot policies that can be deployed zero-shot in the original scene. We find that digital cousin scenes that preserve geometric and semantic affordances can be produced automatically, and can be used to train policies that outperform policies trained on digital twins, achieving 90% vs. 25% success rates under zero-shot sim-to-real transfer. Additional details are available at https://digital-cousins.github.io/.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Oct 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 52; 2026: 31.

88 papers.

Sort:

By area:

  1. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  2. 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 101
  3. 2026RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robotscited by 65
  4. 2025SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agentcited by 53
  5. 2025NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidancecited by 46
  6. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 42
  7. 2025GWM: Towards Scalable Gaussian World Models for Robotic Manipulationcited by 37
  8. 2025EmbodiedGen: Towards a Generative 3D World Engine for Embodied Intelligencecited by 32
  9. 2025MetaScenes: Towards Automated Replica Creation for Real-world 3D Scanscited by 31
  10. 2025Decompositional Neural Scene Reconstruction with Generative Diffusion Priorcited by 30
  11. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 29
  12. 2025DRAWER: Digital Reconstruction and Articulation With Environment Realismcited by 29
38 more
  1. 2025X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Realcited by 25
  2. 2025PIN-WM: Learning Physics-INformed World Models for Non-Prehensile Manipulationcited by 22
  3. 2025FreeArt3D: Training-Free Articulated Object Generation using 3D Diffusioncited by 22
  4. 2024ArtFormer: Controllable Generation of Diverse 3D Articulated Objectscited by 21
  5. 2025LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scanscited by 21
  6. 2025InternScenes: A Large-scale Simulatable Indoor Scene Dataset with Realistic Layoutscited by 18
  7. 2025Robot Learning from a Physical World Modelcited by 18
  8. 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 18
  9. 2025FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulationcited by 17
  10. 2025Compositional Generative Model of Unbounded 4D Citiescited by 16
  11. 2025GSWorld: Closed-Loop Photo-Realistic Simulation Suite for Robotic Manipulationcited by 16
  12. 2025MesaTask: Towards Task-Driven Tabletop Scene Generation via 3D Spatial Reasoningcited by 15
  13. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  14. 2025SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Modelcited by 15
  15. 2025Agentic 3D Scene Generation with Spatially Contextualized VLMscited by 14
  16. 2025OptiScene: LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimizationcited by 14
  17. 2025HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Videocited by 13
  18. 2025LARM: A Large Articulated-Object Reconstruction Modelcited by 12
  19. 2025REALM: A Real-to-Sim Validated Benchmark for Generalization in Robotic Manipulationcited by 12
  20. 2024Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identificationscited by 10
  21. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 10
  22. 2025Position: Foundation Models Need Digital Twin Representationscited by 10
  23. 2025OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learningcited by 10
  24. 2025RoomCraft: Controllable and Complete 3D Indoor Scene Generationcited by 9
  25. 2025SpaceControl: Introducing Test-Time Spatial Control to 3D Generative Modelingcited by 9
  26. 2025ART: Articulated Reconstruction Transformercited by 9
  27. 2026SimRecon: SimReady Compositional Scene Reconstruction from Real Videoscited by 9
  28. 2026AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discoverycited by 9
  29. 2026Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Modelscited by 8
  30. 2024UrbanCAD: Towards Highly Controllable and Photorealistic 3D Vehicles for Urban Scene Simulationcited by 7
  31. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
  32. 2025Robot Learning from Any Imagescited by 7
  33. 2025SIMPACT: Simulation-Enabled Action Planning using Vision-Language Modelscited by 7
  34. 2026FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videoscited by 7
  35. 2025REACT3D: Recovering Articulations for Interactive Physical 3D Scenescited by 6
  36. 2026REST3D: Reconstructing Physically Stable 3D Scenes from a Single Imagecited by 6
  37. 2025DriveE2E: Closed-Loop Benchmark for End-to-End Autonomous Driving through Real-to-Simulationcited by 5
  38. 2025UrbanVerse: Scaling Urban Simulation by Watching City-Tour Videoscited by 5

50 most cited of 88 All 88 (JSON)