mlsanityData to 14 Oct 2026

RAVEL: Rare Concept Generation and Editing via Graph-driven Relational Guidance

arXiv 2412.09614 ↗First posted 12 Dec 2024cs.CV cs.CLcited by 3

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Despite impressive visual fidelity, current text-to-image (T2I) diffusion models struggle to depict rare, complex, or culturally nuanced concepts due to training data limitations. We introduce RAVEL, a training-free framework that significantly improves rare concept generation, context-driven image editing, and self-correction by integrating graph-based retrieval-augmented generation (RAG) into diffusion pipelines. Unlike prior RAG and LLM-enhanced methods reliant on visual exemplars, static captions or pre-trained knowledge of models, RAVEL leverages structured knowledge graphs to retrieve compositional, symbolic, and relational context, enabling nuanced grounding even in the absence of visual priors. To further refine generation quality, we propose SRD, a novel self-correction module that iteratively updates prompts via multi-aspect alignment feedback, enhancing attribute accuracy, narrative coherence, and semantic fidelity. Our framework is model-agnostic and compatible with leading diffusion models including Stable Diffusion XL, Flux, and DALL-E 3. We conduct extensive evaluations across three newly proposed benchmarks - MythoBench, Rare-Concept-1K, and NovelBench. RAVEL also consistently outperforms SOTA methods across perceptual, alignment, and LLM-as-a-Judge metrics. These results position RAVEL as a robust paradigm for controllable and interpretable T2I generation in long-tail domains.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 40 of its 54 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

40 of 54 references.

Closest ancestors · 12

Same direction, newest first

  1. 2023Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Following28 Nov
  2. 2023Self-correcting LLM-controlled Diffusion Models27 Nov
  3. 2023LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts16 Oct
  4. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models23 May
  5. 2023Generating images of rare concepts using pre-trained diffusion models27 Apr
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  7. 2022Investigating Prompt Engineering in Diffusion Models21 Nov
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  9. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models26 Oct
  10. 2022DiffEdit: Diffusion-based semantic image editing with mask guidance20 Oct
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2020Denoising Diffusion Implicit Modelscited by 9,781
  8. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  9. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  11. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  12. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
16 more
  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  2. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  4. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  5. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
  6. 2024Graph Retrieval-Augmented Generation: A Surveycited by 324
  7. 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 218
  8. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 210
  9. 2023HIVE: Harnessing Human Feedback for Instructional Visual Editingcited by 173
  10. 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editingcited by 133
  11. 2024A Closer Look at the Limitations of Instruction Tuningcited by 77
  12. 2023Contrastive Chain-of-Thought Promptingcited by 52
  13. 2024FairRAG: Fair Human Generation via Fair Retrieval Augmentationcited by 34
  14. 2024ReFIR: Grounding Large Restoration Models with Retrieval Augmentationcited by 15
  15. 2023On the Limitation of Diffusion Models for Synthesizing Training Datasetscited by 10
  16. 2023RetriBooru: Leakage-Free Retrieval of Conditions from Reference Images for Subject-Driven Generationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Rare-to-Frequent: Unlocking Compositional Generation Power of Diffusion Models on Rare Concepts with LLM Guidance
  2. 2026Rare Concept Generation via Counterfactual Inference in Diffusion Models
  3. 2025Concept Lancet: Image Editing with Compositional Representation Transplant
  4. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs
  5. 2024SGEdit: Bridging LLM with Text2Image Generative Model for Scene Graph-based Image Editing
  6. 2024Editing Massive Concepts in Text-to-Image Diffusion Models
  7. 2025ACE: Concept Editing in Diffusion Models without Performance Degradation
  8. 2026ADAPT: Attention Driven Adaptive Prompt Scheduling and InTerpolating Orthogonal Complements for Rare Concepts Generation