# Automated Creation of Digital Cousins for Robust Policy Learning

arXiv: https://arxiv.org/abs/2410.07408 · HTML: https://mlsanity.com/paper/2410.07408/

## Key facts

- Cited by 91 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2024.

## Built on by (most cited first, 30 of 91)

- [SAM 3D: 3Dfy Anything in Images](https://mlsanity.com/paper/2511.16624/) (2025)
- [CAST: Component-Aligned 3D Scene Reconstruction from an RGB Image](https://mlsanity.com/paper/2502.12894/) (2025)
- [RoboCasa365: A Large-Scale Simulation Framework for Training and Benchmarking Generalist Robots](https://mlsanity.com/paper/2603.04356/) (2026)
- [SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agent](https://mlsanity.com/paper/2509.20414/) (2025)
- [NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance](https://mlsanity.com/paper/2505.08712/) (2025)
- [Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/2503.24361/) (2025)
- [GWM: Towards Scalable Gaussian World Models for Robotic Manipulation](https://mlsanity.com/paper/2508.17600/) (2025)
- [EmbodiedGen: Towards a Generative 3D World Engine for Embodied Intelligence](https://mlsanity.com/paper/2506.10600/) (2025)
- [BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activities](https://mlsanity.com/paper/2503.05652/) (2025)
- [Decompositional Neural Scene Reconstruction with Generative Diffusion Prior](https://mlsanity.com/paper/2503.14830/) (2025)
- [MetaScenes: Towards Automated Replica Creation for Real-world 3D Scans](https://mlsanity.com/paper/2505.02388/) (2025)
- [DRAWER: Digital Reconstruction and Articulation With Environment Realism](https://mlsanity.com/paper/2504.15278/) (2025)
- [X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Real](https://mlsanity.com/paper/2505.07096/) (2025)
- [ArtFormer: Controllable Generation of Diverse 3D Articulated Objects](https://mlsanity.com/paper/2412.07237/) (2024)
- [PIN-WM: Learning Physics-INformed World Models for Non-Prehensile Manipulation](https://mlsanity.com/paper/2504.16693/) (2025)
- [FreeArt3D: Training-Free Articulated Object Generation using 3D Diffusion](https://mlsanity.com/paper/2510.25765/) (2025)
- [LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans](https://mlsanity.com/paper/2507.02861/) (2025)
- [InternScenes: A Large-scale Simulatable Indoor Scene Dataset with Realistic Layouts](https://mlsanity.com/paper/2509.10813/) (2025)
- [Robot Learning from a Physical World Model](https://mlsanity.com/paper/2511.07416/) (2025)
- [GSWorld: Closed-Loop Photo-Realistic Simulation Suite for Robotic Manipulation](https://mlsanity.com/paper/2510.20813/) (2025)
- [WorldGen: From Text to Traversable and Interactive 3D Worlds](https://mlsanity.com/paper/2511.16825/) (2025)
- [Compositional Generative Model of Unbounded 4D Cities](https://mlsanity.com/paper/2501.08983/) (2025)
- [FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulation](https://mlsanity.com/paper/2505.10075/) (2025)
- [SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Model](https://mlsanity.com/paper/2512.10957/) (2025)
- [MesaTask: Towards Task-Driven Tabletop Scene Generation via 3D Spatial Reasoning](https://mlsanity.com/paper/2509.22281/) (2025)
- [Simulating the Visual World with Artificial Intelligence: A Roadmap](https://mlsanity.com/paper/2511.08585/) (2025)
- [REALM: A Real-to-Sim Validated Benchmark for Generalization in Robotic Manipulation](https://mlsanity.com/paper/2512.19562/) (2025)
- [Agentic 3D Scene Generation with Spatially Contextualized VLMs](https://mlsanity.com/paper/2505.20129/) (2025)
- [OptiScene: LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimization](https://mlsanity.com/paper/2506.07570/) (2025)
- [HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Video](https://mlsanity.com/paper/2510.05560/) (2025)

## Abstract (arXiv)

Training robot policies in the real world can be unsafe, costly, and difficult to scale. Simulation serves as an inexpensive and potentially limitless source of training data, but suffers from the semantics and physics disparity between simulated and real-world environments. These discrepancies can be minimized by training in digital twins, which serve as virtual replicas of a real scene but are expensive to generate and cannot produce cross-domain generalization. To address these limitations, we propose the concept of digital cousins, a virtual asset or scene that, unlike a digital twin, does not explicitly model a real-world counterpart but still exhibits similar geometric and semantic affordances. As a result, digital cousins simultaneously reduce the cost of generating an analogous virtual environment while also facilitating better robustness during sim-to-real domain transfer by providing a distribution of similar training scenes. Leveraging digital cousins, we introduce a novel method for their automated creation, and propose a fully automated real-to-sim-to-real pipeline for generating fully interactive scenes and training robot policies that can be deployed zero-shot in the original scene. We find that digital cousin scenes that preserve geometric and semantic affordances can be produced automatically, and can be used to train policies that outperform policies trained on digital twins, achieving 90% vs. 25% success rates under zero-shot sim-to-real transfer. Additional details are available at https://digital-cousins.github.io/.