# Real2Render2Real: Scaling Robot Data Without Dynamics Simulation or Robot Hardware

arXiv: https://arxiv.org/abs/2505.09601 · HTML: https://mlsanity.com/paper/2505.09601/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2025.

## Built on by (most cited first, 30 of 44)

- [Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey](https://mlsanity.com/paper/2508.13073/) (2025)
- [A Survey on Efficient Vision-Language-Action Models](https://mlsanity.com/paper/2510.24795/) (2025)
- [PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies](https://mlsanity.com/paper/2512.16881/) (2025)
- [Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactions](https://mlsanity.com/paper/2511.04665/) (2025)
- [AnchorDream: Repurposing Video Diffusion for Embodiment-Aware Robot Data Synthesis](https://mlsanity.com/paper/2512.11797/) (2025)
- [Dexterous Manipulation Policies from RGB Human Videos via 3D Hand-Object Trajectory Reconstruction](https://mlsanity.com/paper/2602.09013/) (2026)
- [OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learning](https://mlsanity.com/paper/2512.13100/) (2025)
- [AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discovery](https://mlsanity.com/paper/2605.23204/) (2026)
- [Embodied Foundation Models at the Edge: A Survey of Deployment Constraints and Mitigation Strategies](https://mlsanity.com/paper/2603.16952/) (2026)
- [Reconstruction by Generation: 3D Multi-Object Scene Reconstruction from Sparse Observations](https://mlsanity.com/paper/2604.27106/) (2026)
- [R2RGEN: Real-to-Real 3D Data Generation for Spatially Generalized Manipulation](https://mlsanity.com/paper/2510.08547/) (2025)
- [TwinRL: Digital Twin-Driven Reinforcement Learning for Real-World Robotic Manipulation](https://mlsanity.com/paper/2602.09023/) (2026)
- [Phys2Real: Fusing VLM Priors with Interactive Online Adaptation for Uncertainty-Aware Sim-to-Real Manipulation](https://mlsanity.com/paper/2510.11689/) (2025)
- [ReaDy-Go: Real-to-Sim Dynamic 3D Gaussian Splatting Simulation for Environment-Specific Visual Navigation with Moving Obstacles](https://mlsanity.com/paper/2602.11575/) (2026)
- [A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies](https://mlsanity.com/paper/2604.13645/) (2026)
- [Real2Edit2Real: Generating Robotic Demonstrations via a 3D Control Interface](https://mlsanity.com/paper/2512.19402/) (2025)
- [Scaling Single Human Demonstrations for Imitation Learning using Generative Foundational Models](https://mlsanity.com/paper/2602.12734/) (2026)
- [EmbodiedSWE: Coding Agents for Long Horizon Dexterous Robotics](https://mlsanity.com/paper/2609.27308/) (2026)
- [FAR-Dex: Few-shot Data Augmentation and Adaptive Residual Policy Refinement for Dexterous Manipulation](https://mlsanity.com/paper/2603.10451/) (2026)
- [RoboDream: Compositional World Models for Scalable Robot Data Synthesis](https://mlsanity.com/paper/2606.02577/) (2026)
- [IGFuse: Interactive 3D Gaussian Scene Reconstruction via Multi-Scans Fusion](https://mlsanity.com/paper/2508.13153/) (2025)
- [Self-Augmented Robot Trajectory: Efficient Imitation Learning via Safe Self-augmentation with Demonstrator-annotated Precision](https://mlsanity.com/paper/2509.09893/) (2025)
- [AOMGen: Photoreal, Physics-Consistent Demonstration Generation for Articulated Object Manipulation](https://mlsanity.com/paper/2512.18396/) (2025)
- [ObjSplat: Geometry-Aware Gaussian Surfels for Active Object Reconstruction](https://mlsanity.com/paper/2601.06997/) (2026)
- [MimicIK: Real-Time Generative Inverse Kinematics from Teleoperation with FK Consistency](https://mlsanity.com/paper/2606.15148/) (2026)
- [DynamicManip: Enabling Dynamic Manipulation from a Single Static Demonstration](https://mlsanity.com/paper/2608.01452/) (2026)
- [DREAM: Deployment-Time Demonstration Generation via Real-to-Sim for Scalable Policy Adaptation](https://mlsanity.com/paper/2608.29078/) (2026)
- [DSG-World: Learning a 3D Gaussian World Model from Dual State Videos](https://mlsanity.com/paper/2506.05217/) (2025)
- [3D Generation for Embodied AI and Robotic Simulation: A Survey](https://mlsanity.com/paper/2604.26509/) (2026)
- [GASE: Gaussian Splatting-Based Automated System for Reconstructing Embodied-Simulation Environments](https://mlsanity.com/paper/2606.17520/) (2026)

## Abstract (arXiv)

Scaling robot learning requires vast and diverse datasets. Yet the prevailing data collection paradigm-human teleoperation-remains costly and constrained by manual effort and physical robot access. We introduce Real2Render2Real (R2R2R), a novel approach for generating robot training data without relying on object dynamics simulation or teleoperation of robot hardware. The input is a smartphone-captured scan of one or more objects and a single video of a human demonstration. R2R2R renders thousands of high visual fidelity robot-agnostic demonstrations by reconstructing detailed 3D object geometry and appearance, and tracking 6-DoF object motion. R2R2R uses 3D Gaussian Splatting (3DGS) to enable flexible asset generation and trajectory synthesis for both rigid and articulated objects, converting these representations to meshes to maintain compatibility with scalable rendering engines like IsaacLab but with collision modeling off. Robot demonstration data generated by R2R2R integrates directly with models that operate on robot proprioceptive states and image observations, such as vision-language-action models (VLA) and imitation learning policies. Physical experiments suggest that models trained on R2R2R data from a single human demonstration can match the performance of models trained on 150 human teleoperation demonstrations. Project page: https://real2render2real.com