mlsanityData to 7 Oct 2026

Enhance Image Classification via Inter-Class Image Mixup with Diffusion Model

arXiv 2403.19600 ↗First posted 28 Mar 2024cs.CVcited by 34

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Text-to-image (T2I) generative models have recently emerged as a powerful tool, enabling the creation of photo-realistic images and giving rise to a multitude of applications. However, the effective integration of T2I models into fundamental image classification tasks remains an open question. A prevalent strategy to bolster image classification performance is through augmenting the training set with synthetic images generated by T2I models. In this study, we scrutinize the shortcomings of both current generative and conventional data augmentation techniques. Our analysis reveals that these methods struggle to produce images that are both faithful (in terms of foreground objects) and diverse (in terms of background contexts) for domain-specific concepts. To tackle this challenge, we introduce an innovative inter-class data augmentation method known as Diff-Mix (https://github.com/Zhicaiwww/Diff-Mix), which enriches the dataset by performing image translations between classes. Our empirical results demonstrate that Diff-Mix achieves a better balance between faithfulness and diversity, leading to a marked improvement in performance across diverse image classification scenarios, including few-shot, conventional, and long-tail classifications for domain-specific datasets.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 47 of its 72 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

47 of 72 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023Controlling Text-to-Image Diffusion by Orthogonal Finetuning12 Jun
  2. 2023ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models7 Jun
  3. 2023Synthetic Data from Diffusion Models Improves ImageNet Classification17 Apr
  4. 2023Your Diffusion Model is Secretly a Zero-Shot Classifier28 Mar
  5. 2023Effective Data Augmentation With Diffusion Models7 Feb
  6. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clones16 Dec
  7. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  8. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  9. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  12. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
26 more
  1. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  2. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  3. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  4. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  6. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  7. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  8. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  9. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  10. 2019When Does Label Smoothing Help?cited by 985
  11. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  12. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  13. 2020StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generationcited by 331
  14. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  15. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  16. 2022Novel View Synthesis with Diffusion Modelscited by 299
  17. 2017Data Augmentation Generative Adversarial Networkscited by 278
  18. 2023StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learnerscited by 177
  19. 2022Person Image Synthesis via Denoising Diffusion Modelcited by 104
  20. 2021The Majority Can Help The Minority: Context-rich Minority Oversampling for Long-tailed Classificationcited by 100
  21. 2023Leaving Reality to Imagination: Robust Classification via Generated Datasetscited by 91
  22. 2022Invariant Grounding for Video Question Answeringcited by 68
  23. 2022Incorporating Bias-aware Margins into Contrastive Loss for Collaborative Filteringcited by 29
  24. 2023Toward Understanding Generative Data Augmentationcited by 25
  25. 2023Bi-directional Distribution Alignment for Transductive Zero-Shot Learningcited by 7
  26. 2023Improving Out-of-Distribution Robustness of Classifiers via Generative Interpolationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 19; 2026: 9.

34 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey on Data Augmentationcited by 38
  2. 2024A Survey on Mixup Augmentations and Beyondcited by 18
  3. 2024Advancing Fine-Grained Classification by Structure and Subject Preserving Augmentationcited by 12
  4. 2024Inversion Circle Interpolation: Diffusion-based Image Augmentation for Data-scarce Classificationcited by 11
  5. 2025Diffusion-based Synthetic Data Generation for Visible-Infrared Person Re-Identificationcited by 10
  6. 2025Context-guided Responsible Data Augmentation with Diffusion Modelscited by 9
  7. 2025Generate, Refine, and Encode: Leveraging Synthesized Novel Samples for On-the-Fly Fine-Grained Category Discoverycited by 9
  8. 2025GAITGen: Disentangled Motion-Pathology Impaired Gait Generative Model -- Bringing Motion Generation to the Clinical Domaincited by 7
  9. 2024Decoupled Data Augmentation for Improving Image Classificationcited by 6
  10. 2025MDIT: A Model-free Data Interpolation Method for Diverse Instruction Tuningcited by 3
  11. 2025SGD-Mix: Enhancing Domain-Specific Image Classification with Label-Preserving Data Augmentationcited by 3
  12. 2025ProtoConNet: Prototypical Augmentation and Alignment for Open-Set Few-Shot Image Classificationcited by 3
22 more
  1. 2024MixDiff: Mixing Natural and Synthetic Images for Robust Self-Supervised Representationscited by 2
  2. 2025Beyond One-Hot Labels: Semantic Mixing for Model Calibrationcited by 2
  3. 2025Do We Need All the Synthetic Data? Targeted Image Augmentation via Diffusion Modelscited by 2
  4. 2025Improving Knowledge Distillation Under Unknown Covariate Shift Through Confidence-Guided Data Augmentationcited by 2
  5. 2026On the Difficulty of Learning a Meta-network for Training Data Selectioncited by 2
  6. 2026S2-FracMix: Label-Preserving Self-Saliency Mixup Augmentationcited by 2
  7. 2025Mixup Regularization: A Probabilistic Perspectivecited by 1
  8. 2025Enhancing Generalization in Data-free Quantization via Mixup-class Promptingcited by 1
  9. 2026InstructMixup: Instruction-Guided Salient Patch Editing for Robust Data Augmentationcited by 1
  10. 2025NoiseCutMix: A Novel Data Augmentation Approach by Mixing Estimated Noise in Diffusion Modelscited by 0
  11. 2025Data Augmentation via Latent Diffusion Models for Detecting Smell-Related Objects in Historical Artworkscited by 0
  12. 2025GenView++: Unifying Adaptive Generative Augmentation and Quality-Driven Supervision for Contrastive Representation Learningcited by 0
  13. 2025Diffusion-Classifier Synergy: Reward-Aligned Learning via Mutual Boosting Loop for FSCILcited by 0
  14. 2025Salient Concept-Aware Generative Data Augmentationcited by 0
  15. 2025Beyond Objects: Contextual Synthetic Data Generation for Fine-Grained Classificationcited by 0
  16. 2025SASG-DA: Sparse-Aware Semantic-Guided Diffusion Augmentation For Myoelectric Gesture Recognitioncited by 0
  17. 2026BLENDER: Blended Text Embeddings and Diffusion Residuals for Intra-Class Image Synthesis in Deep Metric Learningcited by 0
  18. 2026Diffusion-Based Data Augmentation for Image Recognition: A Systematic Analysis and Evaluationcited by 0
  19. 2026Learning through Creation: A Hash-Free Framework for On-the-Fly Category Discoverycited by 0
  20. 2026Noise2Map: End-to-End Diffusion Model for Semantic Segmentation and Change Detectioncited by 0
  21. 2026LiBaGS: Lightweight Boundary Gap Synthesis for Targeted Synthetic Data Selectioncited by 0
  22. 2026Steering Diffusion Models via Class-Contrastive Influence for Few-Shot Medical Classificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Discriminative Class Tokens for Text-to-Image Diffusion Models
  2. 2024Dataset Augmentation by Mixing Visual Concepts
  3. 2023Text-to-Image Diffusion Models are Zero-Shot Classifiers
  4. 2023Diversity and Diffusion: Observations on Synthetic Image Distributions with Stable Diffusion
  5. 2025Augmented Conditioning Is Enough For Effective Training Image Generation
  6. 2024DreamDA: Generative Data Augmentation with Diffusion Models
  7. 2025Instructing Text-to-Image Diffusion Models via Classifier-Guided Semantic Optimization
  8. 2024Advances in Diffusion Models for Image Data Augmentation: A Review of Methods, Models, Evaluation Metrics and Future Research Directions