Specifying Object Attributes and Relations in Interactive Scene Generation
Not on the map. Why?
Abstract (arXiv)
We introduce a method for the generation of images from an input scene graph. The method separates between a layout embedding and an appearance embedding. The dual embedding leads to generated images that better match the scene graph, have higher visual quality, and support more complex scene graphs. In addition, the embedding scheme supports multiple and diverse output images per scene graph, which can be further controlled by the user. We demonstrate two modes of per-object control: (i) importing elements from other images, and (ii) navigation in the object space, by selecting an appearance archetype. Our code is publicly available at https://www.github.com/ashual/scene_generation
Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Sep 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 30; 2022: 16; 2023: 15; 2024: 10; 2025: 3; 2026: 2.
76 papers.
Sort:
By area:
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
- 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 258
- 2021Illiterate DALL-E Learns to Composecited by 147
- 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
- 2021Instance-Conditioned GANcited by 119
- 2021Spatial-Temporal Transformer for Dynamic Scene Graph Generationcited by 95
- 2022RelTR: Relation Transformer for Scene Graph Generationcited by 95
- 2021Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphscited by 83
- 2021Learning to Compose Visual Relationscited by 72
- 2022HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoisingcited by 72
- 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
- 2023LayoutDM: Transformer-based Diffusion Model for Layout Generationcited by 57
38 more
- 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
- 2022Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Trainingcited by 56
- 2021Text to Image Generation with Semantic-Spatial Aware GANcited by 53
- 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
- 2022Modeling Image Composition for Complex Scene Generationcited by 43
- 2022SceneComposer: Any-Level Semantic Image Synthesiscited by 43
- 2021Context-Aware Layout to Image Generation with Enhanced Object Appearancecited by 41
- 2023SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesiscited by 41
- 2022RepMix: Representation Mixing for Robust Attribution of Synthesized Imagescited by 39
- 2023SceneHGN: Hierarchical Graph Networks for 3D Indoor Scene Generation with Fine-Grained Geometrycited by 38
- 2021High-Resolution Complex Scene Synthesis with Transformerscited by 31
- 2021Instance-wise Occlusion and Depth Orders in Natural Scenescited by 31
- 2022VAEL: Bridging Variational Autoencoders and Probabilistic Logic Programmingcited by 31
- 2021Adversarial Open Domain Adaptation for Sketch-to-Photo Synthesiscited by 24
- 2021Compositional Transformers for Scene Generationcited by 24
- 2021Unconditional Scene Graph Generationcited by 22
- 2021High Resolution Face Editing with Masked GAN Latent Code Optimizationcited by 21
- 2021Top-N Recommendation with Counterfactual User Preference Simulationcited by 21
- 2023SGRec3D: Self-Supervised 3D Scene Graph Learning via Object-Level Scene Reconstructioncited by 20
- 2023Graph Transformer GANs for Graph-Constrained House Generationcited by 19
- 2022Interactive Image Synthesis with Panoptic Layout Generationcited by 17
- 2021OSCAR-Net: Object-centric Scene Graph Attention for Image Attributioncited by 15
- 2022CoGS: Controllable Generation and Search from Sketch and Stylecited by 14
- 2021House-GAN++: Generative Adversarial Layout Refinement Networkscited by 13
- 2021AttrLostGAN: Attribute Controlled Image Synthesis from Reconfigurable Layout and Stylecited by 11
- 2024Scene Graph Disentanglement and Composition for Generalizable Complex Image Generationcited by 11
- 2022Memory-Driven Text-to-Image Generationcited by 10
- 2021Exploiting Relationship for Complex-scene Image Generationcited by 9
- 2022iPLAN: Interactive and Procedural Layout Planningcited by 8
- 2021MIGS: Meta Image Generation from Scene Graphscited by 7
- 2022GraphMapper: Efficient Visual Navigation by Scene Graph Generationcited by 6
- 2022SynCLay: Interactive Synthesis of Histology Images from Bespoke Cellular Layoutscited by 6
- 2024What Makes a Scene ? Scene Graph-based Evaluation and Feedback for Controllable Generationcited by 6
- 2025Inversion-DPO: Precise and Efficient Post-Training for Diffusion Modelscited by 6
- 2021Tackling the Challenges in Scene Graph Generation with Local-to-Global Interactionscited by 5
- 2023PARASOL: Parametric Style Control for Diffusion Image Synthesiscited by 5
- 2023Controllable Image Generation via Collage Representationscited by 5
- 2024Graph Transformer GANs with Graph Masked Modeling for Architectural Layout Generationcited by 5
50 most cited of 76 All 76 (JSON)
- 2026Generative Floor Plan Design with LLMs via Reinforcement Learning with Verifiable Rewards13 May 2026
- 2026Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformers11 Mar 2026
- 2025Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models14 Jul 2025
- 2025Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models12 May 2025
- 2025STAY Diffusion: Styled Layout Diffusion Model for Diverse Layout-to-Image Generation15 Mar 2025
- 2024What Makes a Scene ? Scene Graph-based Evaluation and Feedback for Controllable Generation23 Nov 2024
- 2024Scene Graph Disentanglement and Composition for Generalizable Complex Image Generation1 Oct 2024
- 2024Automatic Scene Generation: State-of-the-Art Techniques, Models, Datasets, Challenges, and Future Prospects14 Sep 2024
- 2024DStruct2Design: Data and Benchmarks for Data Structure Driven Generative Floor Plan Design22 Jul 2024
- 2024Training-free Subject-Enhanced Attention Guidance for Compositional Text-to-image Generation11 May 2024
- 2024Generated Contents Enrichment6 May 2024
- 2024Image Synthesis with Graph Conditioning: CLIP-Guided Diffusion Models for Scene Graphs25 Jan 2024
38 more
- 2024Text-to-Image Cross-Modal Generation: A Systematic Review21 Jan 2024
- 2024Graph Transformer GANs with Graph Masked Modeling for Architectural Layout Generation15 Jan 2024
- 2024Joint Generative Modeling of Grounded Scene Graphs and Images via Diffusion Models2 Jan 2024
- 2023The Challenges of Image Generation Models in Generating Multi-Component Images22 Nov 2023
- 2023Generating by Understanding: Neural Visual Generation with Logical Symbol Groundings26 Oct 2023
- 2023SGRec3D: Self-Supervised 3D Scene Graph Learning via Object-Level Scene Reconstruction27 Sep 2023
- 2023STUPD: A Synthetic Dataset for Spatial and Temporal Relation Reasoning13 Sep 2023
- 2023Synthesis of Annotated Colorectal Cancer Tissue Images from Gland Layout8 May 2023
- 2023LayoutDM: Transformer-based Diffusion Model for Layout Generation4 May 2023
- 2023SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis28 Apr 2023
- 2023Controllable Image Generation via Collage Representations26 Apr 2023
- 2023SPAN: Learning Similarity between Scene Graphs and Images with Transformers2 Apr 2023
- 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation30 Mar 2023
- 2023Unsupervised Traffic Scene Generation with Synthetic 3D Scene Graphs15 Mar 2023
- 2023Graph Transformer GANs for Graph-Constrained House Generation14 Mar 2023
- 2023PARASOL: Parametric Style Control for Diffusion Image Synthesis11 Mar 2023
- 2023SceneHGN: Hierarchical Graph Networks for 3D Indoor Scene Generation with Fine-Grained Geometry16 Feb 2023
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan 2023
- 2022SynCLay: Interactive Synthesis of Histology Images from Bespoke Cellular Layouts28 Dec 2022
- 2022Graph Neural Networks in Computer Vision -- Architectures, Datasets and Common Approaches20 Dec 2022
- 2022HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoising23 Nov 2022
- 2022SceneComposer: Any-Level Semantic Image Synthesis21 Nov 2022
- 2022Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training21 Nov 2022
- 2022DisPositioNet: Disentangled Pose and Identity in Semantic Image Manipulation10 Nov 2022
- 2022T2CI-GAN: Text to Compressed Image generation using Generative Adversarial Network1 Oct 2022
- 2022Memory-Driven Text-to-Image Generation15 Aug 2022
- 2022RepMix: Representation Mixing for Robust Attribution of Synthesized Images5 Jul 2022
- 2022Modeling Image Composition for Complex Scene Generation2 Jun 2022
- 2022GraphMapper: Efficient Visual Navigation by Scene Graph Generation17 May 2022
- 2022iPLAN: Interactive and Procedural Layout Planning27 Mar 2022
- 2022CoGS: Controllable Generation and Search from Sketch and Style17 Mar 2022
- 2022Interactive Image Synthesis with Panoptic Layout Generation4 Mar 2022
- 2022VAEL: Bridging Variational Autoencoders and Probabilistic Logic Programming7 Feb 2022
- 2022RelTR: Relation Transformer for Scene Graph Generation27 Jan 2022
- 2021Leveraging in-domain supervision for unsupervised image-to-image translation tasks via multi-stream generators30 Dec 2021
- 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editing4 Dec 2021
- 2021Panoptic-aware Image-to-Image Translation3 Dec 2021
- 2021Instance-wise Occlusion and Depth Orders in Natural Scenes29 Nov 2021
50 newest of 76 All 76 (JSON)
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
- 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 258
- 2022HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoisingcited by 72
- 2023LayoutDM: Transformer-based Diffusion Model for Layout Generationcited by 57
- 2023SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesiscited by 41
- 2024Scene Graph Disentanglement and Composition for Generalizable Complex Image Generationcited by 11
- 2025Inversion-DPO: Precise and Efficient Post-Training for Diffusion Modelscited by 6
- 2023PARASOL: Parametric Style Control for Diffusion Image Synthesiscited by 5
- 2024Joint Generative Modeling of Grounded Scene Graphs and Images via Diffusion Modelscited by 4
- 2025STAY Diffusion: Styled Layout Diffusion Model for Diverse Layout-to-Image Generationcited by 3
- 2024Image Synthesis with Graph Conditioning: CLIP-Guided Diffusion Models for Scene Graphscited by 2
- 2026Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformerscited by 0
- 2026Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformers11 Mar 2026
- 2025Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models14 Jul 2025
- 2025STAY Diffusion: Styled Layout Diffusion Model for Diverse Layout-to-Image Generation15 Mar 2025
- 2024Scene Graph Disentanglement and Composition for Generalizable Complex Image Generation1 Oct 2024
- 2024Image Synthesis with Graph Conditioning: CLIP-Guided Diffusion Models for Scene Graphs25 Jan 2024
- 2024Joint Generative Modeling of Grounded Scene Graphs and Images via Diffusion Models2 Jan 2024
- 2023LayoutDM: Transformer-based Diffusion Model for Layout Generation4 May 2023
- 2023SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis28 Apr 2023
- 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation30 Mar 2023
- 2023PARASOL: Parametric Style Control for Diffusion Image Synthesis11 Mar 2023
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan 2023
- 2022HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoising23 Nov 2022
- 2021Illiterate DALL-E Learns to Composecited by 147
- 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
- 2021Instance-Conditioned GANcited by 119
- 2021Spatial-Temporal Transformer for Dynamic Scene Graph Generationcited by 95
- 2022RelTR: Relation Transformer for Scene Graph Generationcited by 95
- 2021Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphscited by 83
- 2021Learning to Compose Visual Relationscited by 72
- 2021SemanticStyleGAN: Learning Compositional Generative Priors for Controllable Image Synthesis and Editingcited by 58
- 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
- 2022Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Trainingcited by 56
- 2021Text to Image Generation with Semantic-Spatial Aware GANcited by 53
- 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
12 most cited of 60 All 60 (JSON)
- 2026Generative Floor Plan Design with LLMs via Reinforcement Learning with Verifiable Rewards13 May 2026
- 2024What Makes a Scene ? Scene Graph-based Evaluation and Feedback for Controllable Generation23 Nov 2024
- 2024Automatic Scene Generation: State-of-the-Art Techniques, Models, Datasets, Challenges, and Future Prospects14 Sep 2024
- 2024DStruct2Design: Data and Benchmarks for Data Structure Driven Generative Floor Plan Design22 Jul 2024
- 2024Training-free Subject-Enhanced Attention Guidance for Compositional Text-to-image Generation11 May 2024
- 2024Generated Contents Enrichment6 May 2024
- 2024Text-to-Image Cross-Modal Generation: A Systematic Review21 Jan 2024
- 2024Graph Transformer GANs with Graph Masked Modeling for Architectural Layout Generation15 Jan 2024
- 2023The Challenges of Image Generation Models in Generating Multi-Component Images22 Nov 2023
- 2023SGRec3D: Self-Supervised 3D Scene Graph Learning via Object-Level Scene Reconstruction27 Sep 2023
- 2023Synthesis of Annotated Colorectal Cancer Tissue Images from Gland Layout8 May 2023
- 2023Controllable Image Generation via Collage Representations26 Apr 2023
12 newest of 60 All 60 (JSON)