mlsanityData to 14 Oct 2026

GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation

arXiv 2306.04607 ↗First posted 7 Jun 2023cs.CV cs.AIcited by 56

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Diffusion models have attracted significant attention due to the remarkable ability to create content and generate data for tasks like image classification. However, the usage of diffusion models to generate the high-quality object detection data remains an underexplored area, where not only image-level perceptual quality but also geometric conditions such as bounding boxes and camera views are essential. Previous studies have utilized either copy-paste synthesis or layout-to-image (L2I) generation with specifically designed modules to encode the semantic layouts. In this paper, we propose the GeoDiffusion, a simple framework that can flexibly translate various geometric conditions into text prompts and empower pre-trained text-to-image (T2I) diffusion models for high-quality detection data generation. Unlike previous L2I methods, our GeoDiffusion is able to encode not only the bounding boxes but also extra geometric conditions such as camera views in self-driving scenes. Extensive experiments demonstrate GeoDiffusion outperforms previous L2I methods while maintaining 4x training time faster. To the best of our knowledge, this is the first work to adopt diffusion models for layout-to-image generation with geometric conditions and demonstrate that L2I-generated images can be beneficial for improving the performance of object detectors.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 44 of its 64 references that are in the map. First posted on arXiv 7 Jun 2023.

Builds on

44 of 64 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023DDP: Diffusion Model for Dense Visual Prediction30 Mar
  2. 2023LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation16 Feb
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  4. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  5. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting13 Dec
  6. 2022DiffusionDet: Diffusion Model for Object Detection17 Nov
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  9. 2021Palette: Image-to-Image Diffusion Models10 Nov

Other references · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  8. 2023Segment Anythingcited by 8,234
  9. 2017Mask R-CNNcited by 7,277
  10. 2022Generative Adversarial Networkscited by 5,627
  11. 2017Neural Discrete Representation Learningcited by 4,712
  12. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
23 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  3. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  4. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  5. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  6. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  7. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 700
  8. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 596
  9. 2017The Effectiveness of Data Augmentation in Image Classification using Deep Learningcited by 370
  10. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 349
  11. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  12. 2022ReCo: Region-Controlled Text-to-Image Generationcited by 179
  13. 2018GAN Augmentation: Augmenting Training Data using Generative Adversarial Networkscited by 107
  14. 2018Modeling Visual Context is Key to Augmenting Object Detection Datasetscited by 106
  15. 2018Image Generation from Layoutcited by 105
  16. 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
  17. 2019Image Synthesis From Reconfigurable Layout and Stylecited by 85
  18. 2023Open-vocabulary Object Segmentation with Diffusion Modelscited by 82
  19. 2021Context-Aware Layout to Image Generation with Enhanced Object Appearancecited by 41
  20. 2021MultiSiam: Self-supervised Multi-instance Siamese Representation Learning for Autonomous Drivingcited by 41
  21. 2023Mixed Autoencoder for Self-supervised Visual Representation Learningcited by 38
  22. 2021High-Resolution Complex Scene Synthesis with Transformerscited by 31
  23. 2022Task-Customized Self-Supervised Pre-training with Scalable Dynamic Routingcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 18; 2025: 22; 2026: 13.

56 papers.

Sort:

By area:

  1. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 347
  2. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  3. 2024MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenescited by 71
  4. 2023DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Controlcited by 45
  5. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 38
  6. 2024Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesiscited by 35
  7. 2024AeroGen: Enhancing Remote Sensing Object Detection with Diffusion-Driven Data Generationcited by 35
  8. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  9. 2024ODGEN: Domain-specific Object Detection Data Generation with Diffusion Modelscited by 17
  10. 2025Domain-RAG: Retrieval-Guided Compositional Image Generation for Cross-Domain Few-Shot Object Detectioncited by 15
  11. 2023Applications of Large Scale Foundation Models for Autonomous Drivingcited by 14
  12. 2024Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignmentcited by 10
38 more
  1. 2025DC-ControlNet: Decoupling Inter- and Intra-Element Conditions in Image Generation with Diffusion Modelscited by 10
  2. 2023Layered Rendering Diffusion Model for Controllable Zero-Shot Image Synthesiscited by 9
  3. 2024CC-Diff: Enhancing Contextual Coherence in Remote Sensing Image Synthesiscited by 8
  4. 2025QuaDreamer: Controllable Panoramic Video Generation for Quadruped Robotscited by 8
  5. 2024Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Modelscited by 7
  6. 2025ReCon: Region-Controllable Data Augmentation with Rectification and Alignment for Object Detectioncited by 6
  7. 2024ZoDi: Zero-Shot Domain Adaptation with Diffusion-Based Image Transfercited by 5
  8. 2025STAGE: A Stream-Centric Generative World Model for Long-Horizon Driving-Scene Simulationcited by 5
  9. 2025Object Fidelity Diffusion for Remote Sensing Image Generationcited by 5
  10. 2025SIGMA-GEN: Structure and Identity Guided Multi-subject Assembly for Image Generationcited by 5
  11. 2024Auto Cherry-Picker: Learning from High-quality Generative Data Driven by Languagecited by 4
  12. 2024Training-Free Layout-to-Image Generation with Marginal Attention Constraintscited by 4
  13. 2025Detail++: Training-Free Detail Enhancer for T2I Diffusion Modelscited by 4
  14. 2025FICGen: Frequency-Inspired Contextual Disentanglement for Layout-driven Degraded Image Generationcited by 4
  15. 2025Unbiased Object Detection Beyond Frequency with Visually Prompted Image Synthesiscited by 4
  16. 2025Gaussian Splatting is an Effective Data Generator for 3D Object Detectioncited by 2
  17. 2025DIVE: Inverting Conditional Diffusion Models for Discriminative Taskscited by 2
  18. 2025Boosting Domain Generalized and Adaptive Detection with Diffusion Models: Fitness, Generalization, and Transferabilitycited by 2
  19. 2025Controllable 3D Placement of Objects with Scene-Aware Diffusion Modelscited by 2
  20. 2025Adapting Vehicle Detectors for Aerial Imagery to Unseen Domains with Weak Supervisioncited by 2
  21. 2026HuiYanEarth-SAR: A Foundation Model for High-Fidelity and Low-Cost Global Remote Sensing Imagery Generationcited by 2
  22. 2026Envisioning Beyond the Few: Disentangled Semantics and Primitives for Few-Shot Atypical Layout-to-Image Generationcited by 2
  23. 2024LocRef-Diffusion:Tuning-Free Layout and Appearance-Guided Generationcited by 1
  24. 2024Paint Outside the Box: Synthesizing and Selecting Training Data for Visual Groundingcited by 1
  25. 2024Diffusion-based Data Augmentation and Knowledge Distillation with Generated Soft Labels Solving Data Scarcity Problems of SAR Oil Spill Segmentationcited by 1
  26. 2025Scene-Aware Location Modeling for Data Augmentation in Automotive Object Detectioncited by 1
  27. 2025Beyond the Seen: Bounded Distribution Estimation for Open-Vocabulary Learningcited by 1
  28. 2026SynthRender and I-AsSET: Open-Source Framework and Dataset for Bidirectional Sim-Real Transfer in Industrial Object Perceptioncited by 1
  29. 2026Visual Prototype Conditioned Focal Region Generation for UAV-Based Object Detectioncited by 1
  30. 2026V2VCrafter: Consistent Street-View Image Generation Across Vehiclescited by 1
  31. 2026One Image is All You Need: Agentic One-Shot Image Generation via Text-Based World Models for Long-Tail Spatial Perceptioncited by 1
  32. 2026To Blend In, First Decouple: Rethinking Camouflage Image Generation via Context-Decoupled Representationscited by 1
  33. 2024Data-driven Camera and Lidar Simulation Models for Autonomous Driving: A Review from Generative Models to Volume Rendererscited by 0
  34. 2024Identification of Fine-grained Systematic Errors via Controlled Scene Generationcited by 0
  35. 2025ComposeAnything: Composite Object Priors for Text-to-Image Generationcited by 0
  36. 2025FLORA: Efficient Synthetic Data Generation for Object Detection in Low-Data Regimes via finetuning Flux LoRAcited by 0
  37. 2025Towards Continual Expansion of Data Coverage: Automatic Text-guided Edge-case Synthesiscited by 0
  38. 2025Taming Generative Synthetic Data for X-ray Prohibited Item Detectioncited by 0

50 most cited of 56 All 56 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Grounding Text-to-Image Diffusion Models for Controlled High-Quality Image Generation
  2. 2023LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts
  3. 2024Urban Scene Diffusion through Semantic Occupancy Map
  4. 2022SpaText: Spatio-Textual Representation for Controllable Image Generation
  5. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation
  6. 2025DriveGEN: Generalized and Robust 3D Detection in Driving via Controllable Text-to-Image Diffusion Generation
  7. 2024DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception
  8. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation