mlsanityData to 14 Oct 2026

Style Ambiguity Loss Using CLIP

arXiv 2410.02055 ↗First posted 2 Oct 2024cs.CVcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

In this work, we explore using the style ambiguity training objective, originally used to approximate creativity, on a diffusion model. However, this objective requires the use of a pretrained classifier and a labeled dataset. We introduce new forms of style ambiguity loss that do not require training a new classifier or a labeled dataset. Instead of using a classifier, we generate centroids in the CLIP embedding space, and images are classified based on their relative distance to said centroids. We find the centroids via K-means clustering of an unlabeled dataset, as well as using text labels to generate CLIP embeddings, to be used as centroids. Code is available at https://github.com/jamesBaker361/clipcreate

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 15 of its 23 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

15 of 23 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug

Other references · 14

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2022Generative Adversarial Networkscited by 5,627
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2013Auto-Encoding Variational Bayescited by 4,053
  8. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  9. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  10. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  11. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
  12. 2015Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Featurecited by 182
2 more
  1. 2017CAN: Creative Adversarial Networks, Generating "Art" by Learning About Styles and Deviating from Style Normscited by 96
  2. 2022CLIP2GAN: Towards Bridging Text with the Latent Space of GANscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Using Multimodal Foundation Models and Clustering for Improved Style Ambiguity Loss
  2. 2025Control-CLIP: Decoupling Category and Style Guidance in CLIP for Specific-Domain Generation
  3. 2024Measuring Style Similarity in Diffusion Models
  4. 2025Leveraging Diffusion Models for Stylization using Multiple Style Images
  5. 2025SCAdapter: Content-Style Disentanglement for Diffusion Style Transfer
  6. 2024ArtWeaver: Advanced Dynamic Style Integration via Diffusion Model
  7. 2023StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Models
  8. 2023Diversity and Diffusion: Observations on Synthetic Image Distributions with Stable Diffusion