mlsanityData to 7 Oct 2026

SceneScape: Text-Driven Consistent Scene Generation

arXiv 2302.01133 ↗First posted 2 Feb 2023cs.CVcited by 160

Not on the map. Why?

Cited by 54 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a method for text-driven perpetual view generation -- synthesizing long-term videos of various scenes solely, given an input text prompt describing the scene and camera poses. We introduce a novel framework that generates such videos in an online fashion by combining the generative power of a pre-trained text-to-image model with the geometric priors learned by a pre-trained monocular depth prediction model. To tackle the pivotal challenge of achieving 3D consistency, i.e., synthesizing videos that depict geometrically-plausible scenes, we deploy an online test-time training to encourage the predicted depth map of the current frame to be geometrically consistent with the synthesized scene. The depth maps are used to construct a unified mesh representation of the scene, which is progressively constructed along the video generation process. In contrast to previous works, which are applicable only to limited domains, our method generates diverse scenes, such as walkthroughs in spaceships, caves, or ice castles.

Key facts. Cited by 160 papers in the map (data through 7 Oct 2026). Builds on 47 of its 76 references that are in the map. First posted on arXiv 2 Feb 2023.

Builds on

47 of 76 references.

References · 47

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2022Generative Adversarial Networkscited by 5,603
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2021Vision Transformers for Dense Predictioncited by 1,789
  9. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  10. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  12. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
35 more
  1. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  2. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  3. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  4. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  5. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  6. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  7. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  8. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  9. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  10. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  11. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  12. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  13. 2022Blended Latent Diffusioncited by 430
  14. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  15. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  16. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  17. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  18. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  19. 2020Softmax Splatting for Video Frame Interpolationcited by 276
  20. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  21. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  22. 2020Consistent Video Depth Estimationcited by 220
  23. 2023Text-To-4D Dynamic Scene Generationcited by 204
  24. 2020Robust Consistent Video Depth Estimationcited by 188
  25. 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
  26. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  27. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  28. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  29. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  30. 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 99
  31. 2021PixelSynth: Generating a 3D-Consistent Experience from a Single Imagecited by 82
  32. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
  33. 2020Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Imagecited by 77
  34. 2021Consistent Depth of Moving Objects in Videocited by 56
  35. 2022Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Imagecited by 55

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 31; 2024: 49; 2025: 55; 2026: 25.

160 papers.

Sort:

By area:

  1. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  2. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
  3. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  4. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  5. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  6. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  7. 2023WonderJourney: Going from Anywhere to Everywherecited by 157
  8. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  9. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  10. 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fieldscited by 122
  11. 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Controlcited by 117
  12. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
38 more
  1. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
  2. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
  3. 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesiscited by 99
  4. 2024I-Design: Personalized LLM Interior Designercited by 87
  5. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
  6. 2024BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolationcited by 81
  7. 2023Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenescited by 75
  8. 2024MultiDiff: Consistent Novel View Synthesis from a Single Imagecited by 67
  9. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65
  10. 2025Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generationcited by 65
  11. 2023The Chosen One: Consistent Characters in Text-to-Image Diffusion Modelscited by 62
  12. 2023NeRFiller: Completing Scenes via Generative 3D Inpaintingcited by 62
  13. 2024Advances in 3D Generation: A Surveycited by 62
  14. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  15. 2023ControlRoom3D: Room Generation using Semantic Proxy Roomscited by 54
  16. 2024LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generationcited by 53
  17. 2023A Recipe for Scaling up Text-to-Video Generation with Text-free Videoscited by 51
  18. 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Textcited by 50
  19. 2024Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understandingcited by 46
  20. 2023Persistent Nature: A Generative Model of Unbounded 3D Worldscited by 44
  21. 2025World-in-World: World Models in a Closed-Loop Worldcited by 44
  22. 2023Text2Immersion: Generative Immersive Scene with 3D Gaussianscited by 43
  23. 2024SceneTeller: Language-to-3D Scene Generationcited by 43
  24. 2024SceneCraft: Layout-Guided 3D Scene Generationcited by 40
  25. 2024Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusioncited by 39
  26. 2023Alchemist: Parametric Control of Material Properties with Diffusion Modelscited by 38
  27. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generationcited by 33
  28. 2024MVGenMaster: Scaling Multi-View Generation from Any Image via 3D Priors Enhanced Diffusion Modelcited by 31
  29. 2024Mixed Diffusion for 3D Indoor Scene Synthesiscited by 30
  30. 2024MVInpainter: Learning Multi-View Consistent Inpainting to Bridge 2D and 3D Editingcited by 30
  31. 2025SynCity: Training-Free Generation of 3D Worldscited by 30
  32. 2025FlashWorld: High-quality 3D Scene Generation within Secondscited by 30
  33. 2023Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Modelscited by 29
  34. 20243D-SceneDreamer: Text-Driven 3D-Consistent Scene Generationcited by 29
  35. 2025ArtiScene: Language-Driven Artistic 3D Scene Generation Through Image Intermediarycited by 27
  36. 2024SyncTweedies: A General Generative Framework Based on Synchronized Diffusionscited by 26
  37. 2024From Sora What We Can See: A Survey of Text-to-Video Generationcited by 26
  38. 2025StarGen: A Spatiotemporal Autoregression Framework with Video Diffusion Model for Scalable and Controllable Scene Generationcited by 26

50 most cited of 160 All 160 (JSON)