mlsanityData to 13 Oct 2026

Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentation

arXiv 2305.16289 ↗First posted 25 May 2023cs.CV cs.AIcited by 100

Not on the map. Why?

Cited by 26 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Many fine-grained classification tasks, like rare animal identification, have limited training data and consequently classifiers trained on these datasets often fail to generalize to variations in the domain like changes in weather or location. As such, we explore how natural language descriptions of the domains seen in training data can be used with large vision models trained on diverse pretraining datasets to generate useful variations of the training data. We introduce ALIA (Automated Language-guided Image Augmentation), a method which utilizes large vision and language models to automatically generate natural language descriptions of a dataset's domains and augment the training data via language-guided image editing. To maintain data integrity, a model trained on the original dataset filters out minimal image edits and those which corrupt class-relevant information. The resulting dataset is visually consistent with the original training data and offers significantly enhanced diversity. We show that ALIA is able to surpasses traditional data augmentation and text-to-image generated data on fine-grained classification tasks, including cases of domain generalization and contextual bias. Code is available at https://github.com/lisadunlap/ALIA.

Key facts. Cited by 100 papers in the map (data through 13 Oct 2026). Builds on 35 of its 44 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

35 of 44 references.

References · 35

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,484
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,170
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,314
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,605
  8. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,882
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,604
  10. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,421
  11. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,266
  12. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,921
23 more
  1. 2013Fine-Grained Visual Classification of Aircraftcited by 1,709
  2. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,371
  3. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,242
  4. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 899
  5. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 645
  6. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  7. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 391
  8. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 327
  9. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  10. 2023Effective Data Augmentation With Diffusion Modelscited by 282
  11. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  12. 2020Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixupcited by 261
  13. 2018GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classificationcited by 242
  14. 2017Generate To Adapt: Aligning Domains using Generative Adversarial Networkscited by 183
  15. 2022BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotationscited by 99
  16. 2021Repurposing GANs for One-shot Semantic Part Segmentationcited by 88
  17. 2022Distilling Model Failures as Directions in Latent Spacecited by 82
  18. 2021AutoMix: Unveiling the Power of Mixup for Stronger Classifierscited by 69
  19. 2021GAN-Supervised Dense Visual Alignmentcited by 61
  20. 2022Shape-Guided Diffusion with Inside-Outside Attentioncited by 45
  21. 2022Using Language to Extend to Unseen Domainscited by 34
  22. 2022On Guiding Visual Attention with Language Specificationcited by 30
  23. 2020Contrastive Examples for Addressing the Tyranny of the Majoritycited by 21

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 13; 2024: 36; 2025: 37; 2026: 14.

100 papers.

Sort:

By area:

  1. 2023Efficient Dataset Distillation via Minimax Diffusioncited by 82
  2. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 60
  3. 2025UniCTokens: Boosting Personalized Understanding and Generation via Unified Concept Tokenscited by 55
  4. 2023VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolutioncited by 52
  5. 2023LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Imagescited by 46
  6. 2024Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verificationcited by 45
  7. 2023DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Controlcited by 44
  8. 2024LLM as Dataset Analyst: Subpopulation Structure Discovery with Large Language Modelcited by 43
  9. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 40
  10. 2023Feedback-guided Data Synthesis for Imbalanced Classificationcited by 33
  11. 2024A Survey on Data Augmentation in Large Model Eracited by 31
  12. 2024Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challengescited by 30
38 more
  1. 2024Beyond Aesthetics: Cultural Competence in Text-to-Image Modelscited by 30
  2. 2024DataDream: Few-shot Guided Dataset Generationcited by 28
  3. 2024DreamDA: Generative Data Augmentation with Diffusion Modelscited by 24
  4. 2024LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Modelscited by 21
  5. 2024Distribution-Aware Data Expansion with Diffusion Modelscited by 20
  6. 2024ODGEN: Domain-specific Object Detection Data Generation with Diffusion Modelscited by 17
  7. 2025Improving the Scaling Laws of Synthetic Data with Deliberate Practicecited by 17
  8. 2024See What LLMs Cannot Answer: A Self-Challenge Framework for Uncovering LLM Weaknessescited by 15
  9. 2026Automated Optimization Modeling via a Localizable Error-Driven Perspectivecited by 14
  10. 2023AI-Generated Images as Data Source: The Dawn of Synthetic Eracited by 13
  11. 2024Advancing Fine-Grained Classification by Structure and Subject Preserving Augmentationcited by 12
  12. 2025DenseGrounding: Improving Dense Language-Vision Semantics for Ego-Centric 3D Visual Groundingcited by 12
  13. 2025EarthSynth: Generating Informative Earth Observation with Diffusion Modelscited by 12
  14. 2024Inversion Circle Interpolation: Diffusion-based Image Augmentation for Data-scarce Classificationcited by 11
  15. 2024Data Augmentation for Image Classification using Generative AIcited by 11
  16. 2025TULIP: Towards Unified Language-Image Pretrainingcited by 11
  17. 2023On the Limitation of Diffusion Models for Synthesizing Training Datasetscited by 10
  18. 2024Human-Guided Image Generation for Expanding Small-Scale Training Image Datasetscited by 10
  19. 2023GeNIe: Generative Hard Negative Images Through Diffusioncited by 8
  20. 2024GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learningcited by 8
  21. 2024Controlled Training Data Generation with Diffusion Modelscited by 8
  22. 2024A Survey of Data Synthesis Approachescited by 8
  23. 2025High-dimensional Analysis of Synthetic Data Selectioncited by 7
  24. 2023Data Optimization in Deep Learning: A Surveycited by 6
  25. 2024Visual Data Diagnosis and Debiasing with Concept Graphscited by 6
  26. 2024Diffusion Curriculum: Synthetic-to-Real Data Curriculum via Image-Guided Diffusioncited by 6
  27. 2025DDB: Diffusion Driven Balancing to Address Spurious Correlationscited by 6
  28. 2025SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutionscited by 6
  29. 2024FDS: Feedback-guided Domain Synthesis with Multi-Source Conditional Diffusion Models for Domain Generalizationcited by 5
  30. 2024Generalize or Detect? Towards Robust Semantic Segmentation Under Multiple Distribution Shiftscited by 5
  31. 2023Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensemblescited by 4
  32. 2025Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMscited by 4
  33. 2025Does Feasibility Matter? Understanding the Impact of Feasibility on Synthetic Training Datacited by 4
  34. 2025BOOD: Boundary-based Out-Of-Distribution Data Generationcited by 4
  35. 2024On Good Practices for Task-Specific Distillation of Large Pretrained Visual Modelscited by 3
  36. 2025DILLEMA: Diffusion and Large Language Models for Multi-Modal Augmentationcited by 3
  37. 2025Training-Free Synthetic Data Generation with Dual IP-Adapter Guidancecited by 3
  38. 2025Synthetic Industrial Object Detection: GenAI vs. Feature-Based Methodscited by 3

50 most cited of 100 All 100 (JSON)