# WorldRover: A Scalable Synthetic Video Data Engine for World Exploration with Rich Annotations

arXiv: https://arxiv.org/abs/2608.15659 · HTML: https://mlsanity.com/paper/2608.15659/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 38 references that are in the map.
- First posted on arXiv 16 Aug 2026.

## Builds on (most cited first, 30 of 35)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking](https://mlsanity.com/paper/2307.15055/) (2023)
- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots](https://mlsanity.com/paper/2310.13724/) (2023)
- [Playing for Benchmarks](https://mlsanity.com/paper/1709.07322/) (2017)
- [Cosmos 3: Omnimodal World Models for Physical AI](https://mlsanity.com/paper/2606.02800/) (2026)
- [DynamicStereo: Consistent Dynamic Depth from Stereo Videos](https://mlsanity.com/paper/2305.02296/) (2023)
- [Infinite Photorealistic Worlds using Procedural Generation](https://mlsanity.com/paper/2306.09310/) (2023)
- [GameFactory: Creating New Games with Generative Interactive Videos](https://mlsanity.com/paper/2501.08325/) (2025)
- [GameGen-X: Interactive Open-world Game Video Generation](https://mlsanity.com/paper/2411.00769/) (2024)
- [MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft](https://mlsanity.com/paper/2504.08388/) (2025)
- [Yume-1.5: A Text-Controlled Interactive World Generation Model](https://mlsanity.com/paper/2512.22096/) (2025)
- [Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World](https://mlsanity.com/paper/1803.08319/) (2018)
- [Sekai: A Video Dataset towards World Exploration](https://mlsanity.com/paper/2506.15675/) (2025)
- [OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling](https://mlsanity.com/paper/2509.12201/) (2025)
- [Infinite Worlds with Versatile Interactions](https://mlsanity.com/paper/2607.07534/) (2026)
- [UnrealCV: Connecting Computer Vision to Unreal Engine](https://mlsanity.com/paper/1609.01326/) (2016)
- [WorldMark: A Unified Benchmark Suite for Interactive Video World Models](https://mlsanity.com/paper/2604.21686/) (2026)
- [UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AI](https://mlsanity.com/paper/2412.20977/) (2024)
- [BEDLAM2.0: Synthetic Humans and Cameras in Motion](https://mlsanity.com/paper/2511.14394/) (2025)
- [WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG](https://mlsanity.com/paper/2603.23497/) (2026)
- [AirSim360: A Panoramic Simulation Platform within Drone View](https://mlsanity.com/paper/2512.02009/) (2025)
- [AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report](https://mlsanity.com/paper/2607.18367/) (2026)

## Built on by (most cited first, 2 of 2)

- [AI for Games in the Foundation Model Era](https://mlsanity.com/paper/2609.16679/) (2026)
- [PanoPed: Beyond Bounding Boxes for Sim-to-Real Panoramic Pedestrian Tracking](https://mlsanity.com/paper/2610.08826/) (2026)

## Abstract (arXiv)

Learning to generate or reconstruct explorable worlds requires video paired with more than RGB: camera motion, scene geometry, temporal correspondence and, for interactive models, control signals. Real capture can provide some of these signals, but dense geometry and long-range correspondence usually rely on estimation or specialised instrumentation. Rendering provides these quantities directly, yet existing synthetic resources rarely combine them on the same frames while also supporting controlled changes of viewpoint and appearance. We introduce WorldRover, a data engine for generating richly annotated, long-range explorations of artist-built environments. At its core, WorldRover-Engine is an Unreal Engine pipeline that executes and offline-renders minute-scale routes while preserving their full trajectories and scene geometry. The same exploration can be replayed from first-person, third-person, and 360 panoramic cameras under different environmental states. Using WorldRover-Engine, we construct WorldRover-10M, whose sequences pair RGB with metric depth, camera trajectories, and trajectory-derived action signals throughout each exploration. Third-person subsets additionally provide dense optical flow, long-range 2D/3D point tracks with visibility, and a character trajectory distinct from the camera trajectory. The engine can render a traversal from first-person, third-person and 360 panoramic viewpoints, under different environmental states or with a neutral white material, while preserving the route and scene geometry. WorldRover therefore turns long-horizon world exploration into a scalable data-generation problem, providing supervision for models that must build, maintain, and revisit coherent representations of an explorable world.