# Text2Street: Controllable Text-to-image Generation for Street Views

arXiv: https://arxiv.org/abs/2402.04504 · HTML: https://mlsanity.com/paper/2402.04504/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 7 Feb 2024.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [Deep Learning for Medical Image Analysis](https://mlsanity.com/paper/1708.08987/) (2017)
- [DRAW: A Recurrent Neural Network For Image Generation](https://mlsanity.com/paper/1502.04623/) (2015)
- [Generating Images from Captions with Attention](https://mlsanity.com/paper/1511.02793/) (2015)
- [Semantic Image Synthesis via Diffusion Models](https://mlsanity.com/paper/2207.00050/) (2022)
- [Learning What and Where to Draw](https://mlsanity.com/paper/1610.02454/) (2016)
- [Street-View Image Generation from a Bird's-Eye View Layout](https://mlsanity.com/paper/2301.04634/) (2023)
- [BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layout](https://mlsanity.com/paper/2308.01661/) (2023)

## Built on by (most cited first, 13 of 13)

- [DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model](https://mlsanity.com/paper/2410.10429/) (2024)
- [Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis](https://mlsanity.com/paper/2506.11526/) (2025)
- [AVD2: Accident Video Diffusion for Accident Video Description](https://mlsanity.com/paper/2502.14801/) (2025)
- [Text-to-Image Synthesis: A Decade Survey](https://mlsanity.com/paper/2411.16164/) (2024)
- [OccTENS: 3D Occupancy World Model via Temporal Next-Scale Prediction](https://mlsanity.com/paper/2509.03887/) (2025)
- [MObI: Multimodal Object Inpainting Using Diffusion Models](https://mlsanity.com/paper/2501.03173/) (2025)
- [TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Models](https://mlsanity.com/paper/2601.08011/) (2026)
- [From Image Generation to Infrastructure Design: a Multi-agent Pipeline for Street Design Generation](https://mlsanity.com/paper/2509.05469/) (2025)
- [Causal-Entity Reflected Egocentric Traffic Accident Video Synthesis](https://mlsanity.com/paper/2506.23263/) (2025)
- [Text2Traffic: A Text-to-Image Generation and Editing Method for Traffic Scenes](https://mlsanity.com/paper/2511.12932/) (2025)
- [LumiGen: An LVLM-Enhanced Iterative Framework for Fine-Grained Text-to-Image Generation](https://mlsanity.com/paper/2508.04732/) (2025)
- [StreetDesignAI: Broadening Designer Perspectives Through Multi-Persona Evaluation of Cycling Infrastructure](https://mlsanity.com/paper/2601.15671/) (2026)
- [Reference-Guided Diffusion Inpainting For Multimodal Counterfactual Generation](https://mlsanity.com/paper/2507.23058/) (2025)

## Abstract (arXiv)

Text-to-image generation has made remarkable progress with the emergence of diffusion models. However, it is still a difficult task to generate images for street views based on text, mainly because the road topology of street scenes is complex, the traffic status is diverse and the weather condition is various, which makes conventional text-to-image models difficult to deal with. To address these challenges, we propose a novel controllable text-to-image framework, named Text2Street. In the framework, we first introduce the lane-aware road topology generator, which achieves text-to-map generation with the accurate road structure and lane lines armed with the counting adapter, realizing the controllable road topology generation. Then, the position-based object layout generator is proposed to obtain text-to-layout generation through an object-level bounding box diffusion strategy, realizing the controllable traffic object layout generation. Finally, the multiple control image generator is designed to integrate the road topology, object layout and weather description to realize controllable street-view image generation. Extensive experiments show that the proposed approach achieves controllable street-view text-to-image generation and validates the effectiveness of the Text2Street framework for street views.