# SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation

arXiv: https://arxiv.org/abs/2606.28276 · HTML: https://mlsanity.com/paper/2606.28276/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2026.

## Built on by (most cited first, 16 of 16)

- [Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents](https://mlsanity.com/paper/2607.19190/) (2026)
- [EmbodiedSWE: Coding Agents for Long Horizon Dexterous Robotics](https://mlsanity.com/paper/2609.27308/) (2026)
- [ARSTAG: An Agentic Real2Sim2Real System for Task-Specific Robot Data Generation](https://mlsanity.com/paper/2609.24563/) (2026)
- [EMPIRIC: Experiment-Driven Learning of Residual World Models for Robot Planning](https://mlsanity.com/paper/2609.35047/) (2026)
- [Self-Evolving Coding Agents: From Digital Programs to Physical-World Intelligence](https://mlsanity.com/paper/2609.35432/) (2026)
- [Agentic RSR: Real-to-Sim-to-Real through Scene Reconstruction and Execution-Grounded Robot Policies](https://mlsanity.com/paper/2610.10479/) (2026)
- [Track, Articulate, Act: Generating Articulation from Casual Human Videos](https://mlsanity.com/paper/2609.19119/) (2026)
- [ReVeal: A Reconstruction-Aware Real-to-Sim Framework for VLA Policy Evaluation](https://mlsanity.com/paper/2609.23910/) (2026)
- [φ-RIE: From Photorealistic Reconstruction to Interactive Environments](https://mlsanity.com/paper/2609.26795/) (2026)
- [WorldAgent: Verification-Guided Agentic Physical World Construction](https://mlsanity.com/paper/2609.33208/) (2026)
- [VAMPS: Visual and Motor Policies from Sampling-Based Planning](https://mlsanity.com/paper/2610.05331/) (2026)
- [AffordCraft: Scalable Construction of Task-Ready Simulation Assets from Single Images](https://mlsanity.com/paper/2610.06643/) (2026)
- [GATOR: Generative and Agentic 3D Object Reconstruction From Casual Images](https://mlsanity.com/paper/2610.11215/) (2026)
- [Splat2Act: From Captured Scenes to Personalized VLA Manipulation](https://mlsanity.com/paper/2610.13945/) (2026)
- [Unravel: Recovering an Interactive 3D Scene from a Single Image](https://mlsanity.com/paper/2610.16146/) (2026)
- [YUBI-Real2Sim: 3DGS Scene Verification for Reliable Policy Evaluation](https://mlsanity.com/paper/2610.17521/) (2026)

## Abstract (arXiv)

Training and evaluating robot policies in the real world is costly and difficult to scale. We introduce SimFoundry, a modular and automated system for zero-shot real-to-sim scene construction from a video. SimFoundry generates sim-ready digital twins and supports object, scene, and task editing, enabling the automated generation of diverse digital cousins: affordance-preserving variations of reconstructed real-world scenes. Policies trained on SimFoundry data transfer zero-shot to challenging real tasks involving multi-step manipulation, articulated object interaction, and bimanual interaction, and its digital cousins (variations of the original scene, objects, and tasks) facilitate generalization to new real-world conditions. Across 7 manipulation tasks and 5 policy architectures, SimFoundry simulation evaluations strongly predict real-world performance, with mean Pearson correlation 0.911 and mean maximum ranking violation 0.018. When evaluating sim-trained policies zero-shot in the real world, policies trained with object, scene, and task cousins in simulation show average task success rate improvements of 17%, 21%, and 40%, respectively. Additional details at https://research.nvidia.com/labs/gear/simfoundry/ .