mlsanityData to 14 Oct 2026

Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion

arXiv 2310.03502 ↗First posted 5 Oct 2023cs.CVcited by 114

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Text-to-image generation is a significant domain in modern computer vision and has achieved substantial improvements through the evolution of generative architectures. Among these, there are diffusion-based models that have demonstrated essential quality enhancements. These models are generally split into two categories: pixel-level and latent-level approaches. We present Kandinsky1, a novel exploration of latent diffusion architecture, combining the principles of the image prior models with latent diffusion techniques. The image prior model is trained separately to map text embeddings to image embeddings of CLIP. Another distinct feature of the proposed model is the modified MoVQ implementation, which serves as the image autoencoder component. Overall, the designed model contains 3.3B parameters. We also deployed a user-friendly demo system that supports diverse generative modes such as text-to-image generation, image fusion, text and image fusion, image variations generation, and text-guided inpainting/outpainting. Additionally, we released the source code and checkpoints for the Kandinsky models. Experimental evaluations demonstrate a FID score of 8.03 on the COCO-30K dataset, marking our model as the top open-source performer in terms of measurable image generation quality.

Key facts. Cited by 114 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 40 of its 44 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

40 of 44 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition24 Jul
  2. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models5 Jul
  3. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  4. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  5. 2022MagicMix: Semantic Mixing with Diffusion Models28 Oct
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  10. 2021Palette: Image-to-Image Diffusion Models10 Nov

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2022Classifier-Free Diffusion Guidancecited by 5,847
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  10. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  11. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  12. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
18 more
  1. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  2. 2022Video Diffusion Modelscited by 2,305
  3. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  4. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  6. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  7. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  8. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  9. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 694
  10. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  11. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  12. 2023Zero-shot Image-to-Image Translationcited by 505
  13. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  14. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
  15. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 331
  16. 2021Conditional Image Generation with Score-Based Diffusion Modelscited by 197
  17. 2023DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusioncited by 167
  18. 2022MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generationcited by 125

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 42; 2025: 46; 2026: 21.

114 papers.

Sort:

By area:

  1. 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusioncited by 202
  2. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
  3. 2024AEROBLADE: Training-Free Detection of Latent Diffusion Images Using Autoencoder Reconstruction Errorcited by 125
  4. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 88
  5. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  6. 2024MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?cited by 66
  7. 2024λ-ECLIPSE: Multi-Concept Personalized Text-to-Image Diffusion Models by Leveraging CLIP Latent Spacecited by 51
  8. 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 48
  9. 2024Margin-aware Preference Optimization for Aligning Diffusion Models without Referencecited by 46
  10. 2025Lumina-mGPT 2.0: Stand-Alone AutoRegressive Image Modelingcited by 37
  11. 2024Aligned Datasets Improve Detection of Latent Diffusion-Generated Imagescited by 36
  12. 2023Black-box Membership Inference Attacks against Fine-tuned Diffusion Modelscited by 35
38 more
  1. 2024MoMA: Multimodal LLM Adapter for Fast Personalized Image Generationcited by 35
  2. 2024T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitchingcited by 33
  3. 2024BrushEdit: All-In-One Image Inpainting and Editingcited by 31
  4. 2024Open-Vocabulary Attention Maps with Token Optimization for Semantic Segmentation in Diffusion Modelscited by 29
  5. 2024Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similaritiescited by 27
  6. 2024How to Trace Latent Generative Model Generated Images without Artificial Watermark?cited by 26
  7. 2024Kandinsky 3: Text-to-Image Synthesis for Multifunctional Generative Frameworkcited by 25
  8. 2023ECLIPSE: A Resource-Efficient Text-to-Image Prior for Image Generationscited by 24
  9. 2023ZeroRF: Fast Sparse View 360deg Reconstruction with Zero Pretrainingcited by 24
  10. 2024A Practitioner's Guide to Continual Multimodal Pretrainingcited by 23
  11. 2024Towards Reliable Verification of Unauthorized Data Usage in Personalized Text-to-Image Diffusion Modelscited by 20
  12. 2024Improving Long-Text Alignment for Text-to-Image Diffusion Modelscited by 18
  13. 2024PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preferencecited by 17
  14. 2024When Worse is Better: Navigating the compression-generation tradeoff in visual tokenizationcited by 15
  15. 2024Scendi Score: Prompt-Aware Diversity Evaluation via Schur Complement of CLIP Embeddingscited by 15
  16. 2023Kiki or Bouba? Sound Symbolism in Vision-and-Language Modelscited by 14
  17. 2024Evaluating and Mitigating IP Infringement in Visual Generative AIcited by 13
  18. 2024DragText: Rethinking Text Embedding in Point-based Image Editingcited by 13
  19. 2024CustomContrast: A Multilevel Contrastive Perspective For Subject-Driven Text-to-Image Customizationcited by 13
  20. 2025PreciseCam: Precise Camera Control for Text-to-Image Generationcited by 13
  21. 2025Ultra-Resolution Adaptation with Easecited by 13
  22. 2025Enhancing Creative Generation on Stable Diffusion-based Modelscited by 13
  23. 2025SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Modelscited by 13
  24. 2025Stay-Positive: A Case for Ignoring Real Image Features in Fake Image Detectioncited by 11
  25. 2025ControlText: Unlocking Controllable Fonts in Multilingual Text Rendering without Font Annotationscited by 10
  26. 2024HFI: A unified framework for training-free detection and implicit watermarking of latent diffusion model generated imagescited by 9
  27. 2025MultiHuman-Testbench: Benchmarking Image Generation for Multiple Humanscited by 8
  28. 2024Detecting Origin Attribution for Text-to-Image Diffusion Modelscited by 7
  29. 2025Distributional Vision-Language Alignment by Cauchy-Schwarz Divergencecited by 7
  30. 2024OmniFusion Technical Reportcited by 6
  31. 2024REVISION: Rendering Tools Enable Spatial Fidelity in Vision-Language Modelscited by 6
  32. 2024Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMscited by 6
  33. 2024Llama Learns to Direct: DirectorLLM for Human-Centric Video Generationcited by 6
  34. 2025Piece it Together: Part-Based Concepting with IP-Priorscited by 6
  35. 2024Layout-to-Image Generation with Localized Descriptions using ControlNet with Cross-Attention Controlcited by 5
  36. 2024Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategiescited by 5
  37. 2024Training-free Subject-Enhanced Attention Guidance for Compositional Text-to-image Generationcited by 5
  38. 2025PIXELS: Progressive Image Xemplar-based Editing with Latent Surgerycited by 5

50 most cited of 114 All 114 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Kandinsky 3.0 Technical Report
  2. 2023Text-driven Visual Synthesis with Latent Diffusion Prior
  3. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
  4. 2023Text-to-image Diffusion Models in Generative AI: A Survey
  5. 2023PRedItOR: Text Guided Image Editing with Diffusion Prior
  6. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation
  7. 2022Shifted Diffusion for Text-to-image Generation
  8. 2023TokenCompose: Text-to-Image Diffusion with Token-level Supervision