mlsanityData to 7 Oct 2026

What do we learn from inverting CLIP models?

arXiv 2403.02580 ↗First posted 5 Mar 2024cs.CV cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

We employ an inversion-based approach to examine CLIP models. Our examination reveals that inverting CLIP models results in the generation of images that exhibit semantic alignment with the specified target prompts. We leverage these inverted images to gain insights into various aspects of CLIP models, such as their ability to blend concepts and inclusion of gender biases. We notably observe instances of NSFW (Not Safe For Work) images during model inversion. This phenomenon occurs even for semantically innocuous prompts, like "a beautiful landscape," as well as for prompts involving the names of celebrities.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 26 of its 31 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

26 of 31 references.

References · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  8. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  9. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  10. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  11. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  12. 2023Erasing Concepts from Diffusion Modelscited by 530
14 more
  1. 2021Image Segmentation Using Text and Image Promptscited by 509
  2. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  3. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  4. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
  5. 2022Red-Teaming the Stable Diffusion Safety Filtercited by 257
  6. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 225
  7. 2015Inverting Visual Representations with Convolutional Networkscited by 168
  8. 2021Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implicationscited by 128
  9. 2022What do Vision Transformers Learn? A Visual Explorationcited by 75
  10. 2023CLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenescited by 55
  11. 2023Into the LAIONs Den: Investigating Hate in Multimodal Datasetscited by 54
  12. 2023Analyzing Bias in Diffusion-based Face Generation Modelscited by 45
  13. 2022Plug-In Inversion: Model-Agnostic Inversion for Vision with Data Augmentationscited by 18
  14. 2023Identifying and Mitigating Model Failures through Few-shot CLIP-aided Diffusion Generationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 2.

13 papers.

Sort:

By area:

  1. 2025X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIPcited by 21
  2. 2024If CLIP Could Talk: Understanding Vision-Language Model Representations Through Their Preferred Concept Descriptionscited by 14
  3. 2025Adversarially Robust CLIP Models Can Induce Better (Robust) Perceptual Metricscited by 10
  4. 2025Implicit Inversion turns CLIP into a Decodercited by 4
  5. 2025LeakyCLIP: Extracting Training Data from CLIPcited by 4
  6. 2024Unaligning Everything: Or Aligning Any Text to Any Image in Multimodal Modelscited by 3
  7. 2025Model Inversion with Layer-Specific Modeling and Alignment for Data-Free Continual Learningcited by 3
  8. 2025Does CLIP perceive art the same way we do?cited by 2
  9. 2025Image Reconstruction as a Tool for Feature Analysiscited by 1
  10. 2025On the dynamic evolution of CLIP texture-shape bias and its relationship to human alignment and model robustnesscited by 1
  11. 2024If you can describe it, they can see it: Cross-Modal Learning of Visual Concepts from Textual Descriptionscited by 0
  12. 2026Multimodal Dataset Distillation Made Simple by Prototype-Guided Data Synthesiscited by 0
1 more
  1. 2026TrustCLIP: Learning Private Visual Features via Adversarial Reconstructioncited by 0