Generating 360deg Video is What You Need For a 3D Scene
Not on the map. Why?
Abstract (arXiv)
Generating 3D scenes is still a challenging task due to the lack of readily available scene data. Most existing methods only produce partial scenes and provide limited navigational freedom. We introduce a practical and scalable solution that uses 360deg video as an intermediate scene representation, capturing the full-scene context and ensuring consistent visual content throughout the generation. We propose WorldPrompter, a generative pipeline that synthesizes traversable 3D scenes from text prompts. WorldPrompter incorporates a conditional 360deg panoramic video generator, capable of producing a 128-frame video that simulates a person walking through and capturing a virtual environment. The resulting video is then reconstructed as Gaussian splats by a fast feedforward 3D reconstructor, enabling a true walkable experience within the 3D scene. Experiments demonstrate that our panoramic video generation model, trained with a mix of image and video data, achieves convincing spatial and temporal consistency for static scenes. This is validated by an average COLMAP matching rate of 94.6%, allowing for high-quality panoramic Gaussian splat reconstruction and improved navigation throughout the scene. Qualitative and quantitative results also show it outperforms the state-of-the-art 360deg video generators and 3D scene generation models.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 41 of its 63 references that are in the map. First posted on arXiv 2 Apr 2025.
Builds on
41 of 63 references.
References · 41
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
29 more
- 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
- 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
- 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
- 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
- 2024gsplat: An Open-Source Library for Gaussian Splattingcited by 279
- 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
- 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
- 2023PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Predictioncited by 165
- 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
- 2024Recent Advances in 3D Gaussian Splattingcited by 130
- 2024Long-LRM: Long-sequence Large Reconstruction Model for Wide-coverage Gaussian Splatscited by 111
- 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
- 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 88
- 2024360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Modelcited by 56
- 20244K4DGen: Panoramic 4D Generation at 4K Resolutioncited by 54
- 2024LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generationcited by 53
- 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Textcited by 50
- 2020EMLight: Lighting Estimation via Spherical Distribution Approximationcited by 47
- 2023Pyramid Diffusion for Fine 3D Large Scene Generationcited by 44
- 2023Text2Immersion: Generative Immersive Scene with 3D Gaussianscited by 43
- 2024DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusioncited by 38
- 2024MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generationcited by 35
- 2023EverLight: Indoor-Outdoor Editable HDR Lighting Estimationcited by 32
- 2024A Construct-Optimize Approach to Sparse View Synthesis without Camera Posecited by 25
- 2022Text2Light: Zero-Shot Text-Driven HDR Panorama Generationcited by 24
- 2024From an Image to a Scene: Learning to Imagine the World from a Million 360 Videoscited by 23
- 2024Vid3D: Synthesis of Dynamic 3D Scenes using 2D Video Diffusioncited by 1
Built on by
Citing papers per year: 2025: 1; 2026: 4.
5 papers.
- 2026OmniRoam: World Wandering via Long-Horizon Panoramic Video Generationcited by 15
- 2026PanoWorld: Real-World Panoramic Generationcited by 3
- 2025CustomX: Unified Character, Action, and Scene Customization in Video World Modelscited by 1
- 2026AlayaVista: Streaming World Modeling from Panoramic States to Perspective Videocited by 1
- 2026Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generationcited by 0
- 2026AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video13 Sep 2026
- 2026PanoWorld: Real-World Panoramic Generation10 Jul 2026
- 2026OmniRoam: World Wandering via Long-Horizon Panoramic Video Generation31 Mar 2026
- 2026Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation14 Mar 2026
- 2025CustomX: Unified Character, Action, and Scene Customization in Video World Models18 Dec 2025