mlsanityData to 7 Oct 2026

KNN-Diffusion: Image Generation via Large-Scale Retrieval

arXiv 2204.02849 ↗First posted 6 Apr 2022cs.CV cs.AI cs.CLcited by 135

Not on the map. Why?

Cited by 61 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent text-to-image models have achieved impressive results. However, since they require large-scale datasets of text-image pairs, it is impractical to train them on new domains where data is scarce or not labeled. In this work, we propose using large-scale retrieval methods, in particular, efficient k-Nearest-Neighbors (kNN), which offers novel capabilities: (1) training a substantially small and efficient text-to-image diffusion model without any text, (2) generating out-of-distribution images by simply swapping the retrieval database at inference time, and (3) performing text-driven local semantic manipulations while preserving object identity. To demonstrate the robustness of our method, we apply our kNN approach on two state-of-the-art diffusion backbones, and show results on several different datasets. As evaluated by human studies and automatic metrics, our method achieves state-of-the-art results compared to existing approaches that train text-to-image generation models using images only (without paired text data)

Key facts. Cited by 135 papers in the map (data through 7 Oct 2026). Builds on 44 of its 71 references that are in the map. First posted on arXiv 6 Apr 2022.

Builds on

44 of 71 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2016Layer Normalizationcited by 5,579
  10. 2017Neural Discrete Representation Learningcited by 4,650
  11. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,276
32 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  2. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  3. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  4. 2017Billion-scale similarity search with GPUscited by 2,477
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  6. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  7. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  8. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  9. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 961
  10. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  11. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  12. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  13. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  14. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  15. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  16. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 640
  17. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  18. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  19. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 434
  20. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  21. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  22. 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 285
  23. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  24. 2018A Note on the Inception Scorecited by 225
  25. 2019Connecting Vision and Language with Localized Narrativescited by 225
  26. 2022Memorizing Transformerscited by 219
  27. 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 175
  28. 2021FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizationcited by 71
  29. 2022CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIPcited by 60
  30. 2020RetrieveGAN: Image Synthesis via Differentiable Patch Retrievalcited by 35
  31. 2021RetrievalFuse: Neural 3D Scene Reconstruction with a Databasecited by 28
  32. 2014Machine learning approach for text and document miningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 18; 2023: 45; 2024: 35; 2025: 25; 2026: 12.

135 papers.

Sort:

By area:

  1. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  2. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  3. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  4. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  5. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  6. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  7. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  8. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  9. 2022Blended Latent Diffusioncited by 430
  10. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  11. 2023Tool Learning with Foundation Modelscited by 302
  12. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
38 more
  1. 2023FreeDoM: Training-Free Energy-Guided Conditional Diffusion Modelcited by 290
  2. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 282
  3. 2023Text-to-image Diffusion Models in Generative AI: A Surveycited by 264
  4. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  5. 2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
  6. 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
  7. 2023Subject-driven Text-to-Image Generation via Apprenticeship Learningcited by 210
  8. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 208
  9. 2023Text-To-4D Dynamic Scene Generationcited by 204
  10. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
  11. 2023UniIR: Training and Benchmarking Universal Multimodal Information Retrieverscited by 165
  12. 2022Retrieval-Augmented Multimodal Language Modelingcited by 125
  13. 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Modelscited by 116
  14. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  15. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  16. 2023Diffusion Models for Reinforcement Learning: A Surveycited by 84
  17. 2022Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Modelscited by 63
  18. 2022Improved Vector Quantized Diffusion Modelscited by 62
  19. 2023The Chosen One: Consistent Characters in Text-to-Image Diffusion Modelscited by 62
  20. 2023Concept Decomposition for Visual Exploration and Inspirationcited by 61
  21. 2023Learning Customized Visual Models with Retrieval-Augmented Knowledgecited by 58
  22. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  23. 2022Efficient Diffusion Models for Vision: A Surveycited by 50
  24. 2023DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Imagecited by 48
  25. 2023Photoswap: Personalized Subject Swapping in Imagescited by 47
  26. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
  27. 2025Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflectioncited by 42
  28. 2024Compositional Text-to-Image Generation with Dense Blob Representationscited by 38
  29. 2022Few-Shot Diffusion Modelscited by 37
  30. 2022Semi-Parametric Neural Image Synthesiscited by 36
  31. 2022Self-Guided Diffusion Modelscited by 36
  32. 2024Consistency-diversity-realism Pareto fronts of conditional image generative modelscited by 35
  33. 2025Efficient Diffusion Models: A Surveycited by 35
  34. 2023Retrieval-Augmented Layout Transformer for Content-Aware Layout Generationcited by 34
  35. 2024Improving Diffusion-Based Image Synthesis with Context Predictioncited by 32
  36. 2023Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Modelcited by 29
  37. 2024M&M VTO: Multi-Garment Virtual Try-On and Editingcited by 29
  38. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29

50 most cited of 135 All 135 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Reusing Computation in Text-to-Image Diffusion for Efficient Generation of Image Sets
  2. 2023Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models
  3. 2026i1: A Simple and Fully Open Recipe for Strong Text-to-Image Models
  4. 2024Diffusion Lens: Interpreting Text Encoders in Text-to-Image Pipelines
  5. 2023Image retrieval outperforms diffusion models on data augmentation
  6. 2022Shifted Diffusion for Text-to-image Generation
  7. 2024Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modeling
  8. 2022Implementing and Experimenting with Diffusion Models for Text-to-Image Generation