mlsanityData to 14 Oct 2026

Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusion

arXiv 2407.13759 ↗First posted 18 Jul 2024cs.CV cs.GRcited by 39

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

We present a method for generating Streetscapes-long sequences of views through an on-the-fly synthesized city-scale scene. Our generation is conditioned by language input (e.g., city name, weather), as well as an underlying map/layout hosting the desired trajectory. Compared to recent models for video generation or 3D view synthesis, our method can scale to much longer-range camera trajectories, spanning several city blocks, while maintaining visual quality and consistency. To achieve this goal, we build on recent work on video diffusion, used within an autoregressive framework that can easily scale to long sequences. In particular, we introduce a new temporal imputation method that prevents our autoregressive approach from drifting from the distribution of realistic city imagery. We train our Streetscapes system on a compelling source of data-posed imagery from Google Street View, along with contextual map data-which allows users to generate city views conditioned on any desired city layout, with controllable camera poses. Please see more results at our project page at https://boyangdeng.com/streetscapes.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 47 of its 69 references that are in the map. First posted on arXiv 18 Jul 2024.

Builds on

47 of 69 references.

References · 47

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2022Classifier-Free Diffusion Guidancecited by 5,847
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  7. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  8. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  9. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  10. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  11. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  12. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
35 more
  1. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
  2. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  3. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  4. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 582
  5. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  6. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  7. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  8. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  9. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 385
  10. 2023Photorealistic Video Generation with Diffusion Modelscited by 306
  11. 2022Novel View Synthesis with Diffusion Modelscited by 300
  12. 20223D Neural Field Generation using Triplane Diffusioncited by 274
  13. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 252
  14. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 243
  15. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 231
  16. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 214
  17. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstructioncited by 180
  18. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 175
  19. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generationcited by 162
  20. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  21. 2023State of the Art on Diffusion Models for Visual Computingcited by 127
  22. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  23. 2023BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layoutcited by 102
  24. 2023Compositional 3D Scene Generation using Locally Conditioned Diffusioncited by 101
  25. 2021GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worldscited by 98
  26. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
  27. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  28. 2023ControlRoom3D: Room Generation using Semantic Proxy Roomscited by 56
  29. 2022DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Modelscited by 51
  30. 2023AutoDecoding Latent 3D Diffusion Modelscited by 50
  31. 2023Persistent Nature: A Generative Model of Unbounded 3D Worldscited by 44
  32. 2023Text2Immersion: Generative Immersive Scene with 3D Gaussianscited by 43
  33. 2024Sora Generates Videos with Stunning Geometrical Consistencycited by 31
  34. 2023UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fieldscited by 26
  35. 2023Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Imagecited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 21; 2026: 13.

39 papers.

Sort:

By area:

  1. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  2. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  3. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 94
  4. 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 52
  5. 2025StarGen: A Spatiotemporal Autoregression Framework with Video Diffusion Model for Scalable and Controllable Scene Generationcited by 26
  6. 20253D Scene Generation: A Surveycited by 24
  7. 2024CrossViewDiff: A Cross-View Diffusion Model for Satellite-to-Street View Synthesiscited by 23
  8. 2025X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllabilitycited by 20
  9. 2025Compositional Generative Model of Unbounded 4D Citiescited by 17
  10. 2025Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagerycited by 17
  11. 2025Sat2City: 3D City Generation from A Single Satellite Image with Cascaded Latent Diffusioncited by 16
  12. 2026MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Enginescited by 16
27 more
  1. 2024Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Modelcited by 14
  2. 2026Grounding World Simulation Models in a Real-World Metropoliscited by 12
  3. 2025Seeing through Satellite Images at Street Viewscited by 9
  4. 2024Proc-GS: Procedural Building Generation for City Assembly with 3D Gaussianscited by 6
  5. 2026Chain of Event-Centric Causal Thought for Physically Plausible Video Generationcited by 6
  6. 2025GPS as a Control Signal for Image Generationcited by 5
  7. 2025LSD-3D: Large-Scale 3D Driving Scene Generation with Geometry Groundingcited by 5
  8. 2025Satellite to GroundScape -- Large-scale Consistent Ground View Generation from Satellite Viewscited by 4
  9. 2025OmniView: An All-Seeing Diffusion Model for 3D and 4D View Synthesiscited by 4
  10. 2024SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blendingcited by 3
  11. 2025MeSS: City Mesh-Guided Outdoor Scene Generation with Cross-View Consistent Diffusioncited by 3
  12. 2025From Image Generation to Infrastructure Design: a Multi-agent Pipeline for Street Design Generationcited by 3
  13. 2026Sat2City v2: Native 3D City Asset Generation from a Single Satellite Imagecited by 3
  14. 2025RGE-GS: Reward-Guided Expansive Driving Scene Reconstruction via Diffusion Priorscited by 2
  15. 2025AutoScape: Geometry-Consistent Long-Horizon Scene Generationcited by 2
  16. 2025LaGen: Towards Autoregressive LiDAR Scene Generationcited by 2
  17. 2025Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections of Imagescited by 1
  18. 2026ScenDi: 3D-to-2D Scene Diffusion Cascades for Urban Generationcited by 1
  19. 2026StreetDesignAI: Broadening Designer Perspectives Through Multi-Persona Evaluation of Cycling Infrastructurecited by 1
  20. 2026DiffPlace: Street View Generation via Place-Controllable Diffusion Model Enhancing Place Recognitioncited by 1
  21. 2026VistaGEN: Consistent Driving Video Generation with Fine-Grained Control Using Multiview Visual-Language Reasoningcited by 1
  22. 2026CityRAG: Stepping Into a City via Spatially-Grounded Video Generationcited by 1
  23. 2025SatDreamer360: Multiview-Consistent Generation of Ground-Level Scenes from Satellite Imagerycited by 0
  24. 2026Cross-View Splatter: Feed-Forward View Synthesis with Georeferenced Imagescited by 0
  25. 2026GeoFidelity-Bench: Evaluating Block-Conditioned Geographic Fidelity in Text-to-Image Street-View Generationcited by 0
  26. 2026Physically Plausible Video Generation via Visual-Semantic Chain-of-Events Conditioningcited by 0
  27. 2026StreetDiff: Multi-view Street Scenes Generation via Cross-view Consistent Multi-view Stable Diffusion with Structure Promptscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Designing streetscapes from street-view imagery using diffusion models
  2. 2025From Satellite to Street: A Hybrid Framework Integrating Stable Diffusion and PanoGAN for Consistent Cross-View Synthesis
  3. 2024Urban Scene Diffusion through Semantic Occupancy Map
  4. 2025Streetscape Analysis with Generative AI (SAGAI): Vision-Language Assessment and Mapping of Urban Scenes
  5. 2024From Bird's-Eye to Street View: Crafting Diverse and Condition-Aligned Images with Latent Diffusion Model
  6. 2024StreetCrafter: Street View Synthesis with Controllable Video Diffusion Models
  7. 2026CrossTimeEdit: A Decade-Spanning Cross-View Dataset and Reward-Guided Editing for Historical Street-View Generation
  8. 2026Effective Multi-sensor Conditioning for Street-view Novel-view Synthesis