mlsanityData to 7 Oct 2026

360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model

arXiv 2401.06578 ↗First posted 12 Jan 2024cs.CVcited by 56

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Panorama video recently attracts more interest in both study and application, courtesy of its immersive experience. Due to the expensive cost of capturing 360-degree panoramic videos, generating desirable panorama videos by prompts is urgently required. Lately, the emerging text-to-video (T2V) diffusion methods demonstrate notable effectiveness in standard video generation. However, due to the significant gap in content and motion patterns between panoramic and standard videos, these methods encounter challenges in yielding satisfactory 360-degree panoramic videos. In this paper, we propose a pipeline named 360-Degree Video Diffusion model (360DVD) for generating 360-degree panoramic videos based on the given prompts and motion conditions. Specifically, we introduce a lightweight 360-Adapter accompanied by 360 Enhancement Techniques to transform pre-trained T2V models for panorama video generation. We further propose a new panorama dataset named WEB360 consisting of panoramic video-text pairs for training 360DVD, addressing the absence of captioned panoramic video datasets. Extensive experiments demonstrate the superiority and effectiveness of 360DVD for panorama video generation. Our project page is at https://akaneqwq.github.io/360DVD/.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 57 of its 65 references that are in the map. First posted on arXiv 12 Jan 2024.

Builds on

57 of 65 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Customizing 360-Degree Panoramas through Text-to-Image Diffusion Models28 Oct
  2. 2023Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Prompts18 Oct
  3. 2023PanoDiffusion: 360-degree Panorama Outpainting via Diffusion6 Jul

Other references · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2022Generative Adversarial Networkscited by 5,603
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  10. 2017Neural Discrete Representation Learningcited by 4,650
  11. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
38 more
  1. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  2. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  3. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  4. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  5. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  6. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  7. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  8. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 589
  9. 2017Generative Adversarial Networks: An Overviewcited by 588
  10. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  11. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  12. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  13. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  14. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  15. 2023FreeDoM: Training-Free Energy-Guided Conditional Diffusion Modelcited by 290
  16. 2019Residual Flows for Invertible Generative Modelingcited by 256
  17. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  18. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 193
  19. 2023Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generationcited by 153
  20. 2023Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generationcited by 137
  21. 2023Make-Your-Video: Customized Video Generation Using Textual and Structural Guidancecited by 129
  22. 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Modelscited by 116
  23. 2023EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protectioncited by 109
  24. 2023DiffCollage: Parallel Generation of Large Content with Diffusion Modelscited by 100
  25. 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesiscited by 99
  26. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  27. 2019COCO-GAN: Generation by Parts via Conditional Coordinatingcited by 67
  28. 2019Boundless: Generative Adversarial Networks for Image Extensioncited by 62
  29. 2021InfinityGAN: Towards Infinite-Pixel Image Synthesiscited by 61
  30. 2022StyleLight: HDR Panorama Generation for Lighting Estimation and Editingcited by 59
  31. 2023LAMP: Learn A Motion Pattern for Few-Shot-Based Video Generationcited by 59
  32. 2021In&Out : Diverse Image Outpainting via GAN Inversioncited by 55
  33. 2023CRoSS: Diffusion Model Makes Controllable, Robust and Secure Image Steganographycited by 50
  34. 2022Diverse Plausible 360-Degree Image Outpainting for Efficient 3DCG Background Creationcited by 47
  35. 2023360-Degree Panorama Generation from Few Unregistered NFoV Imagescited by 42
  36. 2022Deep Learning for Omnidirectional Vision: A Survey and New Perspectivescited by 38
  37. 2022Guided Co-Modulated GAN for 360deg Field of View Extrapolationcited by 32
  38. 2021BIPS: Bi-modal Indoor Panorama Synthesis via Residual Depth-aided Adversarial Learningcited by 26

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 29; 2026: 19.

56 papers.

Sort:

By area:

  1. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  2. 2024MagicTime: Time-lapse Video Generation Models as Metamorphic Simulatorscited by 51
  3. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  4. 2025Matrix-3D: Omnidirectional Explorable 3D World Generationcited by 33
  5. 2024V2A-Mark: Versatile Deep Visual-Audio Watermarking for Manipulation Localization and Copyright Protectioncited by 27
  6. 2024Imagine360: Immersive 360 Video Generation from Perspective Anchorcited by 27
  7. 2024DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenescited by 24
  8. 20253D Scene Generation: A Surveycited by 24
  9. 2025A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Visioncited by 23
  10. 2025PanoWan: Lifting Diffusion Video Generation Models to 360deg with Latitude/Longitude-aware Mechanismscited by 22
  11. 2026OmniRoam: World Wandering via Long-Horizon Panoramic Video Generationcited by 15
  12. 2025Beyond the Frame: Generating 360 Panoramic Videos from Perspective Videoscited by 14
38 more
  1. 2025LuxDiT: Lighting Estimation with Video Diffusion Transformercited by 14
  2. 2025Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Controlcited by 14
  3. 2025VideoPanda: Video Panoramic Diffusion with Multi-view Attentioncited by 13
  4. 2025Omnidirectional Multi-Object Trackingcited by 12
  5. 2025PANORAMA: The Rise of Omnidirectional Vision in the Embodied AI Eracited by 12
  6. 2025DiVE: Efficient Multi-View Driving Scenes Generation Based on Video Diffusion Transformercited by 10
  7. 2025Omni2: Unifying Omnidirectional Image Generation and Editing in an Omni Modelcited by 9
  8. 2025Leader360V: The Large-scale, Real-world 360 Video Dataset for Multi-task Learning in Diverse Environmentcited by 8
  9. 2026CubeComposer: Spatio-Temporal Autoregressive 4K 360deg Video Generation from Perspective Videocited by 8
  10. 2025CamPVG: Camera-Controlled Panoramic Video Generation with Epipolar-Aware Diffusioncited by 7
  11. 2025PanoWorld-X: Generating Explorable Panoramic Worlds via Sphere-Aware Video Diffusioncited by 7
  12. 2026360Anything: Geometry-Free Lifting of Images and Videos to 360degcited by 7
  13. 2024ResVR: Joint Rescaling and Viewport Rendering of Omnidirectional Imagescited by 6
  14. 2024OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generationcited by 5
  15. 2025Generating 360deg Video is What You Need For a 3D Scenecited by 5
  16. 2025PanFlow: Decoupled Motion Control for Panoramic Video Generationcited by 5
  17. 2025MIND-Edit: MLLM Insight-Driven Editing via Language-Vision Projectioncited by 4
  18. 2025OmniX: From Unified Panoramic Generation and Perception to Graphics-Ready 3D Scenescited by 4
  19. 2025GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstructioncited by 4
  20. 2026Pantheon360: Taming Digital Twin Generation via 3D-Aware 360deg Video Diffusioncited by 4
  21. 2025TanDiT: Tangent-Plane Diffusion Transformer for High-Quality 360deg Panorama Generationcited by 3
  22. 20254DVD: Cascaded Dense-view Video Diffusion Model for High-quality 4D Content Generationcited by 3
  23. 2026Rein3D: Reinforced 3D Indoor Scene Generation with Panoramic Video Diffusion Modelscited by 3
  24. 2026MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffoldcited by 3
  25. 2026PanoWorld: Real-World Panoramic Generationcited by 3
  26. 2024Artificial Intelligence for Biomedical Video Generationcited by 2
  27. 2025PanoLora: Bridging Perspective and Panoramic Video Generation with LoRA Adaptationcited by 2
  28. 2025OmniTrack++: Omnidirectional Multi-Object Tracking by Learning Large-FoV Trajectory Feedbackcited by 2
  29. 2025GimbalDiffusion: Gravity-Aware Camera Control for Video Generationcited by 2
  30. 2026PanoWorld: Geometry-Consistent Panoramic Video World Modelingcited by 2
  31. 2026SpheRoPE: Zero-Shot Optimization-Free 360 Panorama Generation with Spherical RoPEcited by 2
  32. 2025Wireless Video Semantic Communication with Decoupled Diffusion Multi-frame Compensationcited by 1
  33. 2026What if? Emulative Simulation with World Models for Situated Reasoningcited by 1
  34. 2026CM-EVS: Sparse Panoramic RGB-D-Pose Data for Complete Scene Coveragecited by 1
  35. 2026ABot-3DWorld 0: A Universal World Model to Explore Any 3D Spacecited by 1
  36. 2025A Survey on Text-Driven 360-Degree Panorama Generationcited by 0
  37. 2025Controllable Audio-Visual Viewpoint Generation from 360deg Spatial Informationcited by 0
  38. 2026Gimbal360: Canonicalizing Planar Diffusion for Spherical Panorama Completioncited by 0

50 most cited of 56 All 56 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models
  2. 2023Diffusion360: Seamless 360 Degree Panoramic Image Generation based on Diffusion Models
  3. 2025HoloTime: Taming Video Diffusion Models for Panoramic 4D Scene Generation
  4. 2024DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion
  5. 2025Restereo: Diffusion stereo video generation and restoration
  6. 2023GD-VDM: Generated Depth for better Diffusion-based Video Generation
  7. 2025CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generation
  8. 2025Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusion