RAVEL: Rare Concept Generation and Editing via Graph-driven Relational Guidance
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Despite impressive visual fidelity, current text-to-image (T2I) diffusion models struggle to depict rare, complex, or culturally nuanced concepts due to training data limitations. We introduce RAVEL, a training-free framework that significantly improves rare concept generation, context-driven image editing, and self-correction by integrating graph-based retrieval-augmented generation (RAG) into diffusion pipelines. Unlike prior RAG and LLM-enhanced methods reliant on visual exemplars, static captions or pre-trained knowledge of models, RAVEL leverages structured knowledge graphs to retrieve compositional, symbolic, and relational context, enabling nuanced grounding even in the absence of visual priors. To further refine generation quality, we propose SRD, a novel self-correction module that iteratively updates prompts via multi-aspect alignment feedback, enhancing attribute accuracy, narrative coherence, and semantic fidelity. Our framework is model-agnostic and compatible with leading diffusion models including Stable Diffusion XL, Flux, and DALL-E 3. We conduct extensive evaluations across three newly proposed benchmarks - MythoBench, Rare-Concept-1K, and NovelBench. RAVEL also consistently outperforms SOTA methods across perceptual, alignment, and LLM-as-a-Judge metrics. These results position RAVEL as a robust paradigm for controllable and interpretable T2I generation in long-tail domains.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 40 of its 54 references that are in the map. First posted on arXiv 12 Dec 2024.
Builds on
40 of 54 references.
Closest ancestors · 12
Same direction, newest first
- 2023Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Following28 Nov
- 2023Self-correcting LLM-controlled Diffusion Models27 Nov
- 2023LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts16 Oct
- 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models23 May
- 2023Generating images of rare concepts using pre-trained diffusion models27 Apr
- 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
- 2022Investigating Prompt Engineering in Diffusion Models21 Nov
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
- 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models26 Oct
- 2022DiffEdit: Diffusion-based semantic image editing with mask guidance20 Oct
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
Other references · 28
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
16 more
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
- 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
- 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 218
- 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 210
- 2023HIVE: Harnessing Human Feedback for Instructional Visual Editingcited by 173
- 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 133
- 2024A Closer Look at the Limitations of Instruction Tuningcited by 77
- 2023Contrastive Chain-of-Thought Promptingcited by 52
- 2024FairRAG: Fair Human Generation via Fair Retrieval Augmentationcited by 34
- 2024ReFIR: Grounding Large Restoration Models with Retrieval Augmentationcited by 15
- 2023On the Limitation of Diffusion Models for Synthesizing Training Datasetscited by 10
- 2023RetriBooru: Leakage-Free Retrieval of Conditions from Reference Images for Subject-Driven Generationcited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.
- 2025LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformerscited by 12
- 2026Bridging the Intention-Expression Gap: Aligning Multi-Dimensional Preferences via Hierarchical Relevance Feedback in Text-to-Image Diffusioncited by 0
- 2026RankT2I: A Submodular Framework for Discovering Interpretable and Diverse Semantics in Text-to-Image Modelscited by 0
- 2026RankT2I: A Submodular Framework for Discovering Interpretable and Diverse Semantics in Text-to-Image Models14 Aug 2026
- 2026Bridging the Intention-Expression Gap: Aligning Multi-Dimensional Preferences via Hierarchical Relevance Feedback in Text-to-Image Diffusion16 Mar 2026
- 2025LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformers29 May 2025
Similar, not cited
Close in wording, no citation either way
- 2024Rare-to-Frequent: Unlocking Compositional Generation Power of Diffusion Models on Rare Concepts with LLM Guidance
- 2026Rare Concept Generation via Counterfactual Inference in Diffusion Models
- 2025Concept Lancet: Image Editing with Compositional Representation Transplant
- 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs
- 2024SGEdit: Bridging LLM with Text2Image Generative Model for Scene Graph-based Image Editing
- 2024Editing Massive Concepts in Text-to-Image Diffusion Models
- 2025ACE: Concept Editing in Diffusion Models without Performance Degradation
- 2026ADAPT: Attention Driven Adaptive Prompt Scheduling and InTerpolating Orthogonal Complements for Rare Concepts Generation