mlsanityData to 7 Oct 2026

Generating 360deg Video is What You Need For a 3D Scene

arXiv 2504.02045 ↗First posted 2 Apr 2025cs.GR cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Generating 3D scenes is still a challenging task due to the lack of readily available scene data. Most existing methods only produce partial scenes and provide limited navigational freedom. We introduce a practical and scalable solution that uses 360deg video as an intermediate scene representation, capturing the full-scene context and ensuring consistent visual content throughout the generation. We propose WorldPrompter, a generative pipeline that synthesizes traversable 3D scenes from text prompts. WorldPrompter incorporates a conditional 360deg panoramic video generator, capable of producing a 128-frame video that simulates a person walking through and capturing a virtual environment. The resulting video is then reconstructed as Gaussian splats by a fast feedforward 3D reconstructor, enabling a true walkable experience within the 3D scene. Experiments demonstrate that our panoramic video generation model, trained with a mix of image and video data, achieves convincing spatial and temporal consistency for static scenes. This is validated by an average COLMAP matching rate of 94.6%, allowing for high-quality panoramic Gaussian splat reconstruction and improved navigation throughout the scene. Qualitative and quantitative results also show it outperforms the state-of-the-art 360deg video generators and 3D scene generation models.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 41 of its 63 references that are in the map. First posted on arXiv 2 Apr 2025.

Builds on

41 of 63 references.

References · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  8. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  9. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  10. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  11. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  12. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
29 more
  1. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  2. 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
  3. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  4. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  5. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  6. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  7. 2024gsplat: An Open-Source Library for Gaussian Splattingcited by 279
  8. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  9. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  10. 2023PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Predictioncited by 165
  11. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
  12. 2024Recent Advances in 3D Gaussian Splattingcited by 130
  13. 2024Long-LRM: Long-sequence Large Reconstruction Model for Wide-coverage Gaussian Splatscited by 111
  14. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
  15. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 88
  16. 2024360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Modelcited by 56
  17. 20244K4DGen: Panoramic 4D Generation at 4K Resolutioncited by 54
  18. 2024LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generationcited by 53
  19. 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Textcited by 50
  20. 2020EMLight: Lighting Estimation via Spherical Distribution Approximationcited by 47
  21. 2023Pyramid Diffusion for Fine 3D Large Scene Generationcited by 44
  22. 2023Text2Immersion: Generative Immersive Scene with 3D Gaussianscited by 43
  23. 2024DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusioncited by 38
  24. 2024MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generationcited by 35
  25. 2023EverLight: Indoor-Outdoor Editable HDR Lighting Estimationcited by 32
  26. 2024A Construct-Optimize Approach to Sparse View Synthesis without Camera Posecited by 25
  27. 2022Text2Light: Zero-Shot Text-Driven HDR Panorama Generationcited by 24
  28. 2024From an Image to a Scene: Learning to Imagine the World from a Million 360 Videoscited by 23
  29. 2024Vid3D: Synthesis of Dynamic 3D Scenes using 2D Video Diffusioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.