# Class-Balancing Diffusion Models

arXiv: https://arxiv.org/abs/2305.00562 · HTML: https://mlsanity.com/paper/2305.00562/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 45 of its 55 references that are in the map.
- First posted on arXiv 30 Apr 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 45)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [Solving Inverse Problems in Medical Imaging with Score-Based Generative Models](https://mlsanity.com/paper/2111.08005/) (2021)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [Differentiable Augmentation for Data-Efficient GAN Training](https://mlsanity.com/paper/2006.10738/) (2020)
- [WebVision Database: Visual Learning and Understanding from Web Data](https://mlsanity.com/paper/1708.02862/) (2017)
- [Diff-TTS: A Denoising Diffusion Model for Text-to-Speech](https://mlsanity.com/paper/2104.01409/) (2021)
- [Feature Space Augmentation for Long-Tailed Data](https://mlsanity.com/paper/2008.03673/) (2020)
- [M2m: Imbalanced Classification via Major-to-minor Translation](https://mlsanity.com/paper/2004.00431/) (2020)

## Built on by (most cited first, 30 of 48)

- [InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation](https://mlsanity.com/paper/2309.06380/) (2023)
- [Image Generation Diversity Issues and How to Tame Them](https://mlsanity.com/paper/2411.16171/) (2024)
- [Self-Guided Generation of Minority Samples Using Diffusion Models](https://mlsanity.com/paper/2407.11555/) (2024)
- [Minority-Focused Text-to-Image Generation via Prompt Optimization](https://mlsanity.com/paper/2410.07838/) (2024)
- [Label-Noise Robust Diffusion Models](https://mlsanity.com/paper/2402.17517/) (2024)
- [TabEBM: A Tabular Data Augmentation Method with Distinct Class-Specific Energy-Based Models](https://mlsanity.com/paper/2409.16118/) (2024)
- [Latent-based Diffusion Model for Long-tailed Recognition](https://mlsanity.com/paper/2404.04517/) (2024)
- [Bayesian Learning-driven Prototypical Contrastive Loss for Class-Incremental Learning](https://mlsanity.com/paper/2405.11067/) (2024)
- [Training Class-Imbalanced Diffusion Model Via Overlap Optimization](https://mlsanity.com/paper/2402.10821/) (2024)
- [DreamInsert: Zero-Shot Image-to-Video Object Insertion from A Single Image](https://mlsanity.com/paper/2503.10342/) (2025)
- [Calibrating Generative Models to Distributional Constraints](https://mlsanity.com/paper/2510.10020/) (2025)
- [TabMDA: Tabular Manifold Data Augmentation for Any Classifier using Transformers with In-context Subsetting](https://mlsanity.com/paper/2406.01805/) (2024)
- [TFG-Flow: Training-free Guidance in Multimodal Generative Flow](https://mlsanity.com/paper/2501.14216/) (2025)
- [Boost-and-Skip: A Simple Guidance-Free Diffusion for Minority Generation](https://mlsanity.com/paper/2502.06516/) (2025)
- [CheXGenBench: A Unified Benchmark For Fidelity, Privacy and Utility of Synthetic Chest Radiographs](https://mlsanity.com/paper/2505.10496/) (2025)
- [Long Tail Image Generation Through Feature Space Augmentation and Iterated Learning](https://mlsanity.com/paper/2405.01705/) (2024)
- [FairSkin: Fair Diffusion for Skin Disease Image Generation](https://mlsanity.com/paper/2410.22551/) (2024)
- [Negative Token Merging: Image-based Adversarial Feature Guidance](https://mlsanity.com/paper/2412.01339/) (2024)
- [DiffuLT: How to Make Diffusion Model Useful for Long-tail Recognition](https://mlsanity.com/paper/2403.05170/) (2024)
- [Causal Disentanglement for Robust Long-tail Medical Image Generation](https://mlsanity.com/paper/2504.14450/) (2025)
- [Score-based generative emulation of impact-relevant Earth system model outputs](https://mlsanity.com/paper/2510.04358/) (2025)
- [An Ordinal Diffusion Model for Generating Medical Images with Different Severity Levels](https://mlsanity.com/paper/2403.00452/) (2024)
- [Large Generative Graph Models](https://mlsanity.com/paper/2406.05109/) (2024)
- [PoGDiff: Product-of-Gaussians Diffusion Models for Imbalanced Text-to-Image Generation](https://mlsanity.com/paper/2502.08106/) (2025)
- [VAP-Diffusion: Enriching Descriptions with MLLMs for Enhanced Medical Image Generation](https://mlsanity.com/paper/2506.23641/) (2025)
- [Imbalance in Balance: Online Concept Balancing in Generation Models](https://mlsanity.com/paper/2507.13345/) (2025)
- [GANsemble for Small and Imbalanced Data Sets: A Baseline for Synthetic Microplastics Data](https://mlsanity.com/paper/2404.07356/) (2024)
- [Offline Diffusion Policy for Multi-User Delay-Constrained Scheduling](https://mlsanity.com/paper/2501.12942/) (2025)
- [Language-Informed Hyperspectral Image Synthesis for Imbalanced-Small Sample Classification via Semi-Supervised Conditional Diffusion Model](https://mlsanity.com/paper/2502.19700/) (2025)
- [Contrastive Conditional-Unconditional Alignment for Long-tailed Diffusion Model](https://mlsanity.com/paper/2507.09052/) (2025)

## Abstract (arXiv)

Diffusion-based models have shown the merits of generating high-quality visual data while preserving better diversity in recent studies. However, such observation is only justified with curated data distribution, where the data samples are nicely pre-processed to be uniformly distributed in terms of their labels. In practice, a long-tailed data distribution appears more common and how diffusion models perform on such class-imbalanced data remains unknown. In this work, we first investigate this problem and observe significant degradation in both diversity and fidelity when the diffusion model is trained on datasets with class-imbalanced distributions. Especially in tail classes, the generations largely lose diversity and we observe severe mode-collapse issues. To tackle this problem, we set from the hypothesis that the data distribution is not class-balanced, and propose Class-Balancing Diffusion Models (CBDM) that are trained with a distribution adjustment regularizer as a solution. Experiments show that images generated by CBDM exhibit higher diversity and quality in both quantitative and qualitative ways. Our method benchmarked the generation results on CIFAR100/CIFAR100LT dataset and shows outstanding performance on the downstream recognition task.