mlsanityData to 7 Oct 2026

An Interpretable Evaluation of Entropy-based Novelty of Generative Models

arXiv 2402.17287 ↗First posted 27 Feb 2024cs.LG cs.CV stat.MLcited by 17

Not on the map. Why?

Abstract (arXiv)

The massive developments of generative model frameworks require principled methods for the evaluation of a model's novelty compared to a reference dataset. While the literature has extensively studied the evaluation of the quality, diversity, and generalizability of generative models, the assessment of a model's novelty compared to a reference model has not been adequately explored in the machine learning community. In this work, we focus on the novelty assessment for multi-modal distributions and attempt to address the following differential clustering task: Given samples of a generative model PG and a reference model Pref, how can we discover the sample types expressed by PG more frequently than in Pref? We introduce a spectral approach to the differential clustering task and propose the Kernel-based Entropic Novelty (KEN) score to quantify the mode-based novelty of PG with respect to Pref. We analyze the KEN score for mixture distributions with well-separable components and develop a kernel-based method to compute the KEN score from empirical data. We support the KEN framework by presenting numerical results on synthetic and real image datasets, indicating the framework's effectiveness in detecting novel modes and comparing generative models. The paper's code is available at: www.github.com/buyeah1109/KEN

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 34 of its 44 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

34 of 44 references.

References · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2013Auto-Encoding Variational Bayescited by 3,980
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  8. 2016Improved Techniques for Training GANscited by 2,938
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2017Improved Training of Wasserstein GANscited by 2,223
  11. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  12. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
22 more
  1. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  2. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  3. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  4. 2016Least Squares Generative Adversarial Networkscited by 1,000
  5. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  6. 2018Self-Attention Generative Adversarial Networkscited by 809
  7. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 562
  8. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  9. 2020Reliable Fidelity and Diversity Metrics for Generative Modelscited by 425
  10. 2018Assessing Generative Models via Precision and Recallcited by 401
  11. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 299
  12. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  13. 2017Geometric GANcited by 232
  14. 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 211
  15. 2021How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Modelscited by 193
  16. 2021Pros and Cons of GAN Evaluation Measures: New Developmentscited by 121
  17. 2021Data-Efficient Instance Generation from Instance Discriminationcited by 58
  18. 2021Rebooting ACGAN: Auxiliary Classifier GANs with Stable Trainingcited by 57
  19. 2020A Non-Parametric Test to Detect Data-Copying in Generative Modelscited by 52
  20. 2022StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesiscited by 38
  21. 2022Rarity Score : A New Metric to Evaluate the Uncommonness of Synthesized Imagescited by 35
  22. 2017Megapixel Size Image Creation using Generative Adversarial Networkscited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 4; 2026: 7.

17 papers.

Sort:

By area:

  1. 2024Towards a Scalable Reference-Free Evaluation of Generative Modelscited by 25
  2. 2024Unveiling Differences in Generative Models: A Scalable Differential Clustering Approachcited by 16
  3. 2024Scendi Score: Prompt-Aware Diversity Evaluation via Schur Complement of CLIP Embeddingscited by 15
  4. 2025SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Scorecited by 14
  5. 2024A Multi-Armed Bandit Approach to Online Selection and Evaluation of Generative Modelscited by 13
  6. 2024Be More Diverse than the Most Diverse: Optimal Mixtures of Generative Models via Mixture-UCB Bandit Algorithmscited by 13
  7. 2024Do Vendi Scores Converge with Finite Samples? Truncated Vendi Score for Finite-Sample Convergence Guaranteescited by 12
  8. 2025Kernel-based Unsupervised Embedding Alignment for Enhanced Visual Representation in Vision-language Modelscited by 12
  9. 2025Towards an Explainable Comparison and Alignment of Feature Embeddingscited by 9
  10. 2026MMD Guidance: Training-Free Distribution Adaptation for Diffusion Models via Maximum Mean Discrepancy Guidancecited by 3
  11. 2026The Maximum von Neumann Entropy Principle: Theory and Applications in Machine Learningcited by 3
  12. 2025Transport Novelty Distance: A Distributional Metric for Evaluating Material Generative Modelscited by 2
5 more
  1. 2026DAK-UCB: Diversity-Aware Prompt Routing for LLMs and Generative Modelscited by 2
  2. 2026KODA: Contrastive Representation Comparison and Alignment for Vision-Language Foundation Modelscited by 2
  3. 2026PromptSplit: Revealing Prompt-Level Disagreement in Generative Modelscited by 1
  4. 2026Mixture-Greedy for Online Generative Model Selection: Is UCB Necessary in Diversity-Aware Multi-Armed Bandits?cited by 1
  5. 2026Exploring More to Solve More: Boosting Diversity in Text Diffusion Models via Entropy-Based Guidancecited by 1