# SimGraph: A Unified Framework for Scene Graph-Based Image Generation and Editing

arXiv: https://arxiv.org/abs/2601.21498 · HTML: https://mlsanity.com/paper/2601.21498/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 29 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Image Generation from Scene Graphs](https://mlsanity.com/paper/1804.01622/) (2018)
- [LEDITS++: Limitless Image Editing using Text-to-Image Models](https://mlsanity.com/paper/2311.16711/) (2023)
- [Semantic Image Manipulation Using Scene Graphs](https://mlsanity.com/paper/2004.03677/) (2020)
- [Learning Canonical Representations for Scene Graph to Image Generation](https://mlsanity.com/paper/1912.07414/) (2019)
- [Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training](https://mlsanity.com/paper/2211.11138/) (2022)
- [EGTR: Extracting Graph from Transformer for Scene Graph Generation](https://mlsanity.com/paper/2404.02072/) (2024)
- [PasteGAN: A Semi-Parametric Method to Generate Image from Scene Graph](https://mlsanity.com/paper/1905.01608/) (2019)
- [VAR-CLIP: Text-to-Image Generator with Visual Auto-Regressive Modeling](https://mlsanity.com/paper/2408.01181/) (2024)
- [SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesis](https://mlsanity.com/paper/2304.14573/) (2023)
- [HIG: Hierarchical Interlacement Graph Approach to Scene Graph Generation in Video Understanding](https://mlsanity.com/paper/2312.03050/) (2023)
- [SG-Adapter: Enhancing Text-to-Image Generation with Scene Graph Guidance](https://mlsanity.com/paper/2405.15321/) (2024)
- [HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipation](https://mlsanity.com/paper/2411.18042/) (2024)
- [CYCLO: Cyclic Graph Transformer Approach to Multi-Object Relationship Modeling in Aerial Videos](https://mlsanity.com/paper/2406.01029/) (2024)
- [SGEdit: Bridging LLM with Text2Image Generative Model for Scene Graph-based Image Editing](https://mlsanity.com/paper/2410.11815/) (2024)
- [Visual-Relation Conscious Image Generation from Structured-Text](https://mlsanity.com/paper/1908.01741/) (2019)
- [Complex Scene Image Editing by Scene Graph Comprehension](https://mlsanity.com/paper/2203.12849/) (2022)
- [SATURN: Autoregressive Image Generation Guided by Scene Graphs](https://mlsanity.com/paper/2508.14502/) (2025)

## Abstract (arXiv)

Recent advancements in Generative Artificial Intelligence (GenAI) have significantly enhanced the capabilities of both image generation and editing. However, current approaches often treat these tasks separately, leading to inefficiencies and challenges in maintaining spatial consistency and semantic coherence between generated content and edits. Moreover, a major obstacle is the lack of structured control over object relationships and spatial arrangements. Scene graph-based methods, which represent objects and their interrelationships in a structured format, offer a solution by providing greater control over composition and interactions in both image generation and editing. To address this, we introduce SimGraph, a unified framework that integrates scene graph-based image generation and editing, enabling precise control over object interactions, layouts, and spatial coherence. In particular, our framework integrates token-based generation and diffusion-based editing within a single scene graph-driven model, ensuring high-quality and consistent results. Through extensive experiments, we empirically demonstrate that our approach outperforms existing state-of-the-art methods.