# Evaluating Gemini Robotics Policies in a Veo World Simulator

arXiv: https://arxiv.org/abs/2512.10675 · HTML: https://mlsanity.com/paper/2512.10675/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 11 Dec 2025.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 24 of 24)

- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [Evaluating Real-World Robot Manipulation Policies in Simulation](https://mlsanity.com/paper/2405.05941/) (2024)
- [Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions](https://mlsanity.com/paper/2309.07875/) (2023)
- [Ctrl-World: A Controllable Generative World Model for Robot Manipulation](https://mlsanity.com/paper/2510.10125/) (2025)
- [Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation](https://mlsanity.com/paper/2403.03949/) (2024)
- [Training Agents Inside of Scalable World Models](https://mlsanity.com/paper/2509.24527/) (2025)
- [ALOHA Unleashed: A Simple Recipe for Robot Dexterity](https://mlsanity.com/paper/2410.13126/) (2024)
- [THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation](https://mlsanity.com/paper/2402.08191/) (2024)
- [Gemini Robotics 1.5: Pushing the Frontier of Generalist Robots with Advanced Embodied Reasoning, Thinking, and Motion Transfer](https://mlsanity.com/paper/2510.03342/) (2025)
- [Gemini Robotics: Bringing AI into the Physical World](https://mlsanity.com/paper/2503.20020/) (2025)
- [WorldGym: World Model as An Environment for Policy Evaluation](https://mlsanity.com/paper/2506.00613/) (2025)
- [ALOHA 2: An Enhanced Low-Cost Hardware for Bimanual Teleoperation](https://mlsanity.com/paper/2405.02292/) (2024)
- [Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis](https://mlsanity.com/paper/2506.11526/) (2025)
- [Generating Robot Constitutions & Benchmarks for Semantic Safety](https://mlsanity.com/paper/2503.08663/) (2025)
- [RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluation](https://mlsanity.com/paper/2507.00435/) (2025)
- [Predictive Red Teaming: Breaking Policies Without Breaking Robots](https://mlsanity.com/paper/2502.06575/) (2025)
- [Can AI Perceive Physical Danger and Intervene?](https://mlsanity.com/paper/2509.21651/) (2025)
- [Reliable and Scalable Robot Policy Evaluation with Imperfect Simulators](https://mlsanity.com/paper/2510.04354/) (2025)
- [Building a Credible Case for Safety: Waymo's Approach for the Determination of Absence of Unreasonable Risk](https://mlsanity.com/paper/2306.01917/) (2023)
- [Determining Absence of Unreasonable Risk: Approval Guidelines for an Automated Driving System Deployment](https://mlsanity.com/paper/2505.09880/) (2025)
- [Modeling the Impact of Timeline Algorithms on Opinion Dynamics Using Low-rank Updates](https://mlsanity.com/paper/2402.10053/) (2024)

## Built on by (most cited first, 30 of 51)

- [Causal World Modeling for Robot Control](https://mlsanity.com/paper/2601.21998/) (2026)
- [mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs](https://mlsanity.com/paper/2512.15692/) (2025)
- [DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos](https://mlsanity.com/paper/2602.06949/) (2026)
- [StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing](https://mlsanity.com/paper/2604.05014/) (2026)
- [PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation](https://mlsanity.com/paper/2601.03782/) (2026)
- [World Action Models: The Next Frontier in Embodied AI](https://mlsanity.com/paper/2605.12090/) (2026)
- [World Model for Robot Learning: A Comprehensive Survey](https://mlsanity.com/paper/2605.00080/) (2026)
- [Native Video-Action Pretraining for Generalizable Robot Control](https://mlsanity.com/paper/2607.08639/) (2026)
- [PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies](https://mlsanity.com/paper/2512.16881/) (2025)
- [Interactive World Simulator for Robot Policy Training and Evaluation](https://mlsanity.com/paper/2603.08546/) (2026)
- [LLM-as-a-Verifier: A General-Purpose Verification Framework](https://mlsanity.com/paper/2607.05391/) (2026)
- [RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulation](https://mlsanity.com/paper/2604.19092/) (2026)
- [PlayWorld: Learning Robot World Models from Autonomous Play](https://mlsanity.com/paper/2603.09030/) (2026)
- [World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry](https://mlsanity.com/paper/2604.01985/) (2026)
- [FutureVLA: Joint Visuomotor Prediction for Vision-Language-Action Model](https://mlsanity.com/paper/2603.10712/) (2026)
- [Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence](https://mlsanity.com/paper/2607.07675/) (2026)
- [EVA: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards](https://mlsanity.com/paper/2603.17808/) (2026)
- [Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibration](https://mlsanity.com/paper/2603.06001/) (2026)
- [Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulation](https://mlsanity.com/paper/2603.16669/) (2026)
- [World Action Models: A Survey](https://mlsanity.com/paper/2606.20781/) (2026)
- [Thinking in Frames: How Visual Context and Test-Time Scaling Empower Video Reasoning](https://mlsanity.com/paper/2601.21037/) (2026)
- [How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Position](https://mlsanity.com/paper/2606.15032/) (2026)
- [CoF-T2I: Video Models as Pure Visual Reasoners for Text-to-Image Generation](https://mlsanity.com/paper/2601.10061/) (2026)
- [Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Models](https://mlsanity.com/paper/2602.12628/) (2026)
- [Demonstration-Free Robotic Control via LLM Agents](https://mlsanity.com/paper/2601.20334/) (2026)
- [NovaPlan: Zero-Shot Long-Horizon Manipulation via Closed-Loop Video Language Planning](https://mlsanity.com/paper/2602.20119/) (2026)
- [SC3-Eval: Evaluating Robot Foundation Models via Self-Consistent Video Generation](https://mlsanity.com/paper/2606.18610/) (2026)
- [VAG: Dual-Stream Video-Action Generation for Embodied Data Synthesis](https://mlsanity.com/paper/2604.09330/) (2026)
- [Data Pyramid for Embodied Manipulation: A Survey](https://mlsanity.com/paper/2607.24744/) (2026)
- [RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation](https://mlsanity.com/paper/2607.01060/) (2026)

## Abstract (arXiv)

Generative world models hold significant potential for simulating interactions with visuomotor policies in varied environments. Frontier video models can enable generation of realistic observations and environment interactions in a scalable and general manner. However, the use of video models in robotics has been limited primarily to in-distribution evaluations, i.e., scenarios that are similar to ones used to train the policy or fine-tune the base video model. In this report, we demonstrate that video models can be used for the entire spectrum of policy evaluation use cases in robotics: from assessing nominal performance to out-of-distribution (OOD) generalization, and probing physical and semantic safety. We introduce a generative evaluation system built upon a frontier video foundation model (Veo). The system is optimized to support robot action conditioning and multi-view consistency, while integrating generative image-editing and multi-view completion to synthesize realistic variations of real-world scenes along multiple axes of generalization. We demonstrate that the system preserves the base capabilities of the video model to enable accurate simulation of scenes that have been edited to include novel interaction objects, novel visual backgrounds, and novel distractor objects. This fidelity enables accurately predicting the relative performance of different policies in both nominal and OOD conditions, determining the relative impact of different axes of generalization on policy performance, and performing red teaming of policies to expose behaviors that violate physical or semantic safety constraints. We validate these capabilities through 1600+ real-world evaluations of eight Gemini Robotics policy checkpoints and five tasks for a bimanual manipulator.