mlsanityData to 7 Oct 2026

Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search

arXiv 2102.01645 ↗First posted 2 Feb 2021cs.NE cs.AI cs.LGcited by 57

Not on the map. Why?

Abstract (arXiv)

In this research work we present CLIP-GLaSS, a novel zero-shot framework to generate an image (or a caption) corresponding to a given caption (or image). CLIP-GLaSS is based on the CLIP neural network, which, given an image and a descriptive caption, provides similar embeddings. Differently, CLIP-GLaSS takes a caption (or an image) as an input, and generates the image (or the caption) whose CLIP embedding is the most similar to the input one. This optimal image (or caption) is produced via a generative network, after an exploration by a genetic algorithm. Promising results are shown, based on the experimentation of the image Generators BigGAN and StyleGAN2, and of the text Generator GPT2

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Builds on 14 of its 19 references that are in the map. First posted on arXiv 2 Feb 2021.

Builds on

14 of 19 references.

References · 14

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  8. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  9. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  10. 2013Zero-Shot Learning Through Cross-Modal Transfercited by 237
  11. 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
  12. 2015Generating Images from Captions with Attentioncited by 135
2 more
  1. 2016Learning Visual N-Grams from Web Datacited by 95
  2. 2018An Introductory Survey on Attention Mechanisms in NLP Problemscited by 32

Built on by

Cited by year

Citing papers per year: 2021: 12; 2022: 13; 2023: 20; 2024: 8; 2025: 4.

57 papers.

Sort:

By area:

  1. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  4. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  5. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 186
  6. 2021LAFITE: Towards Language-Free Training for Text-to-Image Generationcited by 126
  7. 2021CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIPcited by 88
  8. 2021Differentiable Quality Diversitycited by 71
  9. 2021FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizationcited by 71
  10. 2023Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Featurescited by 52
  11. 2023MindDiffuser: Controlled Image Reconstruction from Human Brain Activity with Semantic and Structural Diffusioncited by 47
  12. 2023RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Modelcited by 47
38 more
  1. 2023Understanding Multimodal Contrastive Learning and Incorporating Unpaired Datacited by 43
  2. 2021Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Conceptscited by 40
  3. 2023AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Modelcited by 37
  4. 2022A Study on the Evaluation of Generative Modelscited by 35
  5. 2021Reading Isn't Believing: Adversarial Attacks On Multi-Modal Neuronscited by 29
  6. 2022StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translationcited by 28
  7. 2024Towards Explainable, Safe Autonomous Driving with Language Embeddings for Novelty Identification and Active Learning: Framework and Experimental Analysis with Real-World Data Setscited by 25
  8. 2021Semantic Segmentation In-the-Wild Without Seeing Any Segmentation Examplescited by 24
  9. 2025Rethinking Cross-Modal Interaction in Multimodal Diffusion Transformerscited by 19
  10. 2023MaskDiffusion: Boosting Text-to-Image Consistency with Conditional Maskcited by 17
  11. 2025Highly Compressed Tokenizer Can Generate Without Trainingcited by 17
  12. 2022Large-scale Bilingual Language-Image Contrastive Learningcited by 12
  13. 2024Training-free Zero-shot Composed Image Retrieval via Weighted Modality Fusion and Similaritycited by 12
  14. 2022What is Where by Looking: Weakly-Supervised Open-World Phrase-Grounding without Text Inputscited by 11
  15. 2023VGSG: Vision-Guided Semantic-Group Network for Text-based Person Searchcited by 11
  16. 2021AffectGAN: Affect-Based Generative Art Driven by Semanticscited by 10
  17. 2022Arbitrary Style Guidance for Enhanced Diffusion-Based Text-to-Image Generationcited by 9
  18. 2022Traditional Classification Neural Networks are Good Generators: They are Competitive with DDPMs and GANscited by 9
  19. 2024Text-guided Explorable Image Super-resolutioncited by 9
  20. 2022End-to-End Visual Editing with a Generatively Pre-Trained Artistcited by 7
  21. 2023RoCOCO: Robustness Benchmark of MS-COCO to Stress-test Image-Text Matching Modelscited by 7
  22. 2023PaintSeg: Training-free Segmentation via Paintingcited by 7
  23. 2023Safeguarding Data in Multimodal AI: A Differentially Private Approach to CLIP Trainingcited by 7
  24. 2023Prefix-diffusion: A Lightweight Diffusion Model for Diverse Image Captioningcited by 7
  25. 2025Multi-modal contrastive learning adapts to intrinsic dimensions of shared latent variablescited by 7
  26. 2024Posterior sampling via Langevin dynamics based on generative priorscited by 5
  27. 2023IPA-CLIP: Integrating Phonetic Priors into Vision and Language Pretrainingcited by 4
  28. 2023ECO: Ensembling Context Optimization for Vision-Language Modelscited by 4
  29. 2023Text-guided Image-and-Shape Editing and Generation: A Short Surveycited by 3
  30. 2023Reference-based Restoration of Digitized Analog Videotapescited by 3
  31. 2022Are metrics measuring what they should? An evaluation of image captioning task metricscited by 2
  32. 2023Completing Visual Objects via Bridging Generation and Segmentationcited by 2
  33. 2024Contrastive Abstraction for Reinforcement Learningcited by 2
  34. 2022Deep Learning and Synthetic Mediacited by 1
  35. 2023GlyphDiffusion: Text Generation as Image Generationcited by 1
  36. 2025Studying Classifier(-Free) Guidance From a Classifier-Centric Perspectivecited by 1
  37. 2022LDEdit: Towards Generalized Text Guided Image Manipulation via Latent Diffusion Modelscited by 0
  38. 2022Towards Real-Time Text2Video via CLIP-Guided, Pixel-Level Optimizationcited by 0

50 most cited of 57 All 57 (JSON)