mlsanityData to 7 Oct 2026

RevCD -- Reversed Conditional Diffusion for Generalized Zero-Shot Learning

arXiv 2409.00511 ↗First posted 31 Aug 2024cs.CVcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

In Generalized Zero-Shot Learning (GZSL), we aim to recognize both seen and unseen categories using a model trained only on seen categories. In computer vision, this translates into a classification problem, where knowledge from seen categories is transferred to unseen categories by exploiting the relationships between visual features and available semantic information, such as text corpora or manual annotations. However, learning this joint distribution is costly and requires one-to-one training with corresponding semantic information. We present a reversed conditional Diffusion-based model (RevCD) that mitigates this issue by generating semantic features synthesized from visual inputs by leveraging Diffusion models' conditional mechanisms. Our RevCD model consists of a cross Hadamard-Addition embedding of a sinusoidal time schedule and a multi-headed visual transformer for attention-guided embeddings. The proposed approach introduces three key innovations. First, we reverse the process of generating semantic space based on visual data, introducing a novel loss function that facilitates more efficient knowledge transfer. Second, we apply Diffusion models to zero-shot learning - a novel approach that exploits their strengths in capturing data complexity. Third, we demonstrate our model's performance through a comprehensive cross-dataset evaluation. The complete code will be available on GitHub.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 36 of its 55 references that are in the map. First posted on arXiv 31 Aug 2024.

Builds on

36 of 55 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Synthetic Data from Diffusion Models Improves ImageNet Classification17 Apr
  2. 2023Your Diffusion Model is Secretly a Zero-Shot Classifier28 Mar
  3. 2023Text-to-Image Diffusion Models are Zero-Shot Classifiers27 Mar
  4. 2023Diversity is Definitely Needed: Improving Model-Agnostic Zero-shot Classification via Stable Diffusion7 Feb

Other references · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Classifier-Free Diffusion Guidancecited by 5,717
  6. 2016Improved Techniques for Training GANscited by 2,938
  7. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  8. 2017Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Uglycited by 536
  9. 2022Understanding Diffusion Models: A Unified Perspectivecited by 337
  10. 2017Feature Generating Networks for Zero-Shot Learningcited by 257
  11. 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 254
  12. 2013Zero-Shot Learning Through Cross-Modal Transfercited by 237
20 more
  1. 2018Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoderscited by 206
  2. 2015Label-Embedding for Image Classificationcited by 164
  3. 2020A Review of Generalized Zero-Shot Learning Methodscited by 159
  4. 2019Seeing What a GAN Cannot Generatecited by 148
  5. 2016Latent Embeddings for Zero-shot Classificationcited by 129
  6. 2019Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapsecited by 120
  7. 2021Contrastive Embedding for Generalized Zero-Shot Learningcited by 94
  8. 2023Robust Classification via a Single Diffusion Modelcited by 79
  9. 2017A Generative Model For Zero Shot Learning Using Conditional Variational Autoencoderscited by 64
  10. 2018Generative Dual Adversarial Network for Generalized Zero-shot Learningcited by 61
  11. 2021Semantics Disentangling for Generalized Zero-Shot Learningcited by 56
  12. 2017A Simple Exponential Family Framework for Zero-Shot Learningcited by 38
  13. 2017Generating Visual Representations for Zero-Shot Classificationcited by 37
  14. 2023Learning Attention as Disentangler for Compositional Zero-shot Learningcited by 29
  15. 2020Rethinking Generative Zero-Shot Learning: An Ensemble Learning Perspective for Recognising Visual Patchescited by 27
  16. 2022Learning Attention Propagation for Compositional Zero-Shot Learningcited by 20
  17. 2021Generative Multi-Label Zero-Shot Learningcited by 19
  18. 2023Distilled Reverse Attention Network for Open-world Compositional Zero-Shot Learningcited by 11
  19. 2020Generalized Zero-Shot Domain Adaptation via Coupled Conditional Variational Autoencoderscited by 5
  20. 2023An Integral Projection-based Semantic Autoencoder for Zero-Shot Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ZeroDiff++: Substantial Unseen Visual-semantic Correlation in Zero-shot Learning
  2. 2024ZeroMamba: Exploring Visual State Space Model for Zero-Shot Learning
  3. 2024Discriminative Image Generation with Diffusion Models for Zero-Shot Learning
  4. 2026DIFFCZSL: Compositional Zero-Shot Learning Regularized by Diffusion Representations
  5. 2024A Simple and Efficient Baseline for Zero-Shot Generative Classification
  6. 2024Zero-Shot Video Semantic Segmentation based on Pre-Trained Diffusion Models
  7. 2024textitRevelio: Interpreting and leveraging semantic information in diffusion models
  8. 2023VGDiffZero: Text-to-image Diffusion Models Can Be Zero-shot Visual Grounders