mlsanityData to 7 Oct 2026

Image Generation from Scene Graphs

arXiv 1804.01622 ↗First posted 4 Apr 2018cs.CV cs.LGcited by 324

Not on the map. Why?

Cited by 42 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

To truly understand the visual world our models should be able not only to recognize images but also generate them. To this end, there has been exciting recent progress on generating images from natural language descriptions. These methods give stunning results on limited domains such as descriptions of birds or flowers, but struggle to faithfully reproduce complex sentences with many objects and relationships. To overcome this limitation we propose a method for generating images from scene graphs, enabling explicitly reasoning about objects and their relationships. Our model uses graph convolution to process input graphs, computes a scene layout by predicting bounding boxes and segmentation masks for objects, and converts the layout to an image with a cascaded refinement network. The network is trained adversarially against a pair of discriminators to ensure realistic outputs. We validate our approach on Visual Genome and COCO-Stuff, where qualitative results, ablations, and user studies demonstrate our method's ability to generate complex images with multiple objects.

Key facts. Cited by 324 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 79; 2022: 63; 2023: 69; 2024: 47; 2025: 37; 2026: 29.

324 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  2. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  3. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  4. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  5. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
  6. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 258
  7. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 242
  8. 2021Neural Articulated Radiance Fieldcited by 196
  9. 2022ReCo: Region-Controlled Text-to-Image Generationcited by 177
  10. 2023Compositional Chain-of-Thought Prompting for Large Multimodal Modelscited by 160
  11. 2021SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequencescited by 159
  12. 2021Equivariant Subgraph Aggregation Networkscited by 154
38 more
  1. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognitioncited by 151
  2. 2021Illiterate DALL-E Learns to Composecited by 147
  3. 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
  4. 2021Generative Adversarial Transformerscited by 120
  5. 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
  6. 2021Instance-Conditioned GANcited by 119
  7. 2022Panoptic Scene Graph Generationcited by 114
  8. 2024InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Priorcited by 110
  9. 2021Paint by Wordcited by 107
  10. 2021Energy-Based Learning for Scene Graph Generationcited by 104
  11. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
  12. 2021Spatial-Temporal Transformer for Dynamic Scene Graph Generationcited by 95
  13. 2022RelTR: Relation Transformer for Scene Graph Generationcited by 95
  14. 2021CanvasVAE: Learning to Generate Vector Graphic Documentscited by 93
  15. 2023LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generationcited by 92
  16. 2022Touch and Go: Learning from Human-Collected Vision and Touchcited by 84
  17. 2021Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphscited by 83
  18. 2024Multi-LoRA Composition for Image Generationcited by 80
  19. 2023SODA: Bottleneck Diffusion Models for Representation Learningcited by 76
  20. 2021GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Drivingcited by 75
  21. 2022Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesiscited by 75
  22. 2021Object-Region Video Transformerscited by 73
  23. 2022Scene Graph Generation: A Comprehensive Surveycited by 73
  24. 2021Learning to Compose Visual Relationscited by 72
  25. 2022HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoisingcited by 72
  26. 2023Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Modelscited by 72
  27. 2022Teaching Structured Vision&Language Concepts to Vision&Language Modelscited by 71
  28. 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 66
  29. 2021Learning of Visual Relations: The Devil is in the Tailscited by 65
  30. 2023LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generationcited by 65
  31. 2021Learning to Generate Scene Graph from Natural Language Supervisioncited by 64
  32. 2021Improving Text-to-Image Synthesis Using Contrastive Learningcited by 62
  33. 2022DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesiscited by 61
  34. 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
  35. 2021Fully Convolutional Scene Graph Generationcited by 57
  36. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  37. 2022Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Trainingcited by 56
  38. 2022RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detectioncited by 54

50 most cited of 324 All 324 (JSON)