# FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Prediction

arXiv: https://arxiv.org/abs/2509.21657 · HTML: https://mlsanity.com/paper/2509.21657/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 52 of its 55 references that are in the map.
- First posted on arXiv 25 Sep 2025.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)

## Builds on (most cited first, 30 of 52)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [Continuous 3D Perception Model with Persistent State](https://mlsanity.com/paper/2501.12387/) (2025)
- [DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision](https://mlsanity.com/paper/2312.16256/) (2023)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion](https://mlsanity.com/paper/2410.03825/) (2024)
- [ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis](https://mlsanity.com/paper/2409.02048/) (2024)
- [Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass](https://mlsanity.com/paper/2501.13928/) (2025)
- [GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control](https://mlsanity.com/paper/2503.03751/) (2025)
- [Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation](https://mlsanity.com/paper/2312.02145/) (2023)
- [GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image](https://mlsanity.com/paper/2403.12013/) (2024)
- [3D Reconstruction with Spatial Memory](https://mlsanity.com/paper/2408.16061/) (2024)
- [Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image](https://mlsanity.com/paper/2012.09855/) (2020)
- [AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Views](https://mlsanity.com/paper/2505.23716/) (2025)
- [Video Depth Anything: Consistent Depth Estimation for Super-Long Videos](https://mlsanity.com/paper/2501.12375/) (2025)
- [DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos](https://mlsanity.com/paper/2409.02095/) (2024)
- [WonderWorld: Interactive 3D Scene Generation from a Single Image](https://mlsanity.com/paper/2406.09394/) (2024)
- [4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling](https://mlsanity.com/paper/2311.17984/) (2023)
- [FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views](https://mlsanity.com/paper/2502.12138/) (2025)
- [Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval](https://mlsanity.com/paper/2506.03141/) (2025)
- [MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motion](https://mlsanity.com/paper/2409.19152/) (2024)
- [WonderJourney: Going from Anywhere to Everywhere](https://mlsanity.com/paper/2312.03884/) (2023)
- [WorldScore: A Unified Evaluation Benchmark for World Generation](https://mlsanity.com/paper/2504.00983/) (2025)

## Built on by (most cited first, 30 of 38)

- [Lyra 2.0: Explorable Generative 3D Worlds](https://mlsanity.com/paper/2604.13036/) (2026)
- [WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation](https://mlsanity.com/paper/2605.25874/) (2026)
- [VGGRPO: Towards World-Consistent Video Generation with 4D Latent Reward](https://mlsanity.com/paper/2603.26599/) (2026)
- [INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling](https://mlsanity.com/paper/2604.07209/) (2026)
- [Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models](https://mlsanity.com/paper/2603.22212/) (2026)
- [Grounding World Simulation Models in a Real-World Metropolis](https://mlsanity.com/paper/2603.15583/) (2026)
- [MemoBench: Benchmarking World Modeling in Dynamically Changing Environments](https://mlsanity.com/paper/2606.27537/) (2026)
- [World-R1: Reinforcing 3D Constraints for Text-to-Video Generation](https://mlsanity.com/paper/2604.24764/) (2026)
- [Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends](https://mlsanity.com/paper/2606.01164/) (2026)
- [CineScene: Implicit 3D as Effective Scene Representation for Cinematic Video Generation](https://mlsanity.com/paper/2602.06959/) (2026)
- [VIGOR: VIdeo Geometry-Oriented Reward for Temporal Generative Alignment](https://mlsanity.com/paper/2603.16271/) (2026)
- [VideoWeave: Unlocking Geometric Consistency in Video Generation via Joint Geometry-Video Modeling](https://mlsanity.com/paper/2606.14162/) (2026)
- [Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective](https://mlsanity.com/paper/2604.14025/) (2026)
- [WorldReel: 4D Video Generation with Consistent Geometry and Motion Modeling](https://mlsanity.com/paper/2512.07821/) (2025)
- [Warp-as-History: Generalizable Camera-Controlled Video Generation from One Training Video](https://mlsanity.com/paper/2605.15182/) (2026)
- [PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Space](https://mlsanity.com/paper/2607.05373/) (2026)
- [WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity](https://mlsanity.com/paper/2608.02603/) (2026)
- [HarnessEval-W: Agentifying the Evaluation of Visual Worlds](https://mlsanity.com/paper/2608.16859/) (2026)
- [MagicWorld: Towards Long-Horizon Stability for Interactive Video World Exploration](https://mlsanity.com/paper/2511.18886/) (2025)
- [OneWorld: Taming Scene Generation with 3D Unified Representation Autoencoder](https://mlsanity.com/paper/2603.16099/) (2026)
- [GTA: Advancing Image-to-3D World Generation via Geometry Then Appearance Video Diffusion](https://mlsanity.com/paper/2605.12957/) (2026)
- [Drift-Resistant Navigation World Model with Anchored Epipolar Guidance](https://mlsanity.com/paper/2605.24761/) (2026)
- [Envision4D: Envisioning Visual Futures via Feed-forward 4D Gaussian Splatting for Autonomous Driving](https://mlsanity.com/paper/2606.10656/) (2026)
- [OctWorld: Long-Range World-Consistent Video Generation with Octree-Based 3D Mapping](https://mlsanity.com/paper/2609.03919/) (2026)
- [Precise Editing and Flexible Referencing for Interactable Worlds](https://mlsanity.com/paper/2609.34470/) (2026)
- [Envision: Embodied Visual Planning via Goal-Imagery Video Diffusion](https://mlsanity.com/paper/2512.22626/) (2025)
- [SceneExpander: Text-Guided 3D Scene Expansion via Free-Form View Insertion](https://mlsanity.com/paper/2603.27084/) (2026)
- [MMDiff: Extending Diffusion Transformers for Multi-Modal Generation](https://mlsanity.com/paper/2606.16673/) (2026)
- [RoGe: Novel View Synthesis via End-to-End Implicit Reconstruction and Generation](https://mlsanity.com/paper/2609.02847/) (2026)
- [Stabilizing Camera-Controlled Novel View Synthesis at Inference Time](https://mlsanity.com/paper/2609.03639/) (2026)

## Abstract (arXiv)

High-quality 3D world models are pivotal for embodied intelligence and Artificial General Intelligence (AGI), underpinning applications such as AR/VR content creation and robotic navigation. Despite the established strong imaginative priors, current video foundation models lack explicit 3D grounding capabilities, thus being limited in both spatial consistency and their utility for downstream 3D reasoning tasks. In this work, we present FantasyWorld, a geometry-enhanced framework that augments frozen video foundation models with a trainable geometric branch, enabling joint modeling of video latents and an implicit 3D field in a single forward pass. Our approach introduces cross-branch supervision, where geometry cues guide video generation and video priors regularize 3D prediction, thus yielding consistent and generalizable 3D-aware video representations. Notably, the resulting latents from the geometric branch can potentially serve as versatile representations for downstream 3D tasks such as novel view synthesis and navigation, without requiring per-scene optimization or fine-tuning. Extensive experiments show that FantasyWorld effectively bridges video imagination and 3D perception, outperforming recent geometry-consistent baselines in multi-view coherence and style consistency. Ablation studies further confirm that these gains stem from the unified backbone and cross-branch information exchange.