mlsanityData to 14 Oct 2026

Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentation

arXiv 2309.14303 ↗First posted 25 Sep 2023cs.CVcited by 112

Not on the map. Why?

Cited by 50 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Preparing training data for deep vision models is a labor-intensive task. To address this, generative models have emerged as an effective solution for generating synthetic data. While current generative models produce image-level category labels, we propose a novel method for generating pixel-level semantic segmentation labels using the text-to-image generative model Stable Diffusion (SD). By utilizing the text prompts, cross-attention, and self-attention of SD, we introduce three new techniques: class-prompt appending, class-prompt cross-attention, and self-attention exponentiation. These techniques enable us to generate segmentation maps corresponding to synthetic images. These maps serve as pseudo-labels for training semantic segmenters, eliminating the need for labor-intensive pixel-wise annotation. To account for the imperfections in our pseudo-labels, we incorporate uncertainty regions into the segmentation, allowing us to disregard loss from those regions. We conduct evaluations on two datasets, PASCAL VOC and MSCOCO, and our approach significantly outperforms concurrent work. Our benchmarks and code will be released at https://github.com/VinAIResearch/Dataset-Diffusion

Key facts. Cited by 112 papers in the map (data through 14 Oct 2026). Builds on 47 of its 57 references that are in the map. First posted on arXiv 25 Sep 2023.

Builds on

47 of 57 references.

References · 47

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  11. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  12. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
35 more
  1. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  2. 2016Pyramid Scene Parsing Networkcited by 2,479
  3. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  4. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  5. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  6. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
  7. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  8. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  9. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  10. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  11. 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 554
  12. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 499
  13. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  14. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 327
  15. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Modelscited by 280
  16. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  17. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 236
  18. 2021Diffusion Models for Implicit Image Segmentation Ensemblescited by 198
  19. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 172
  20. 2023DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Modelscited by 162
  21. 2023Fair Diffusion: Instructing Text-to-Image Generation Models on Fairnesscited by 151
  22. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  23. 2022BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotationscited by 99
  24. 2023Leaving Reality to Imagination: Robust Classification via Generated Datasetscited by 91
  25. 2021Repurposing GANs for One-shot Semantic Part Segmentationcited by 88
  26. 2023The Bias Amplification Paradox in Text-to-Image Generationcited by 70
  27. 2023DiffusionSeg: Adapting Diffusion Towards Unsupervised Object Discoverycited by 54
  28. 2020Object Segmentation Without Labels with Large-Scale Generative Modelscited by 52
  29. 2022DiffusionInst: Diffusion Model for Instance Segmentationcited by 49
  30. 2021Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Modelscited by 44
  31. 2023Diffusion Models and Semi-Supervised Learners Benefit Mutually with Few Labelscited by 37
  32. 2023Semantic Diffusion Network for Semantic Segmentationcited by 30
  33. 2023LD-ZNet: A Latent Diffusion Approach for Text-Based Image Segmentationcited by 29
  34. 2023MaskDiff: Modeling Mask Distribution with Diffusion Probabilistic Model for Few-Shot Instance Segmentationcited by 18
  35. 2022Cap2Aug: Caption guided Image to Image data Augmentationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 42; 2025: 42; 2026: 22.

112 papers.

Sort:

By area:

  1. 2024A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)cited by 58
  2. 2024SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentationcited by 41
  3. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  4. 2024InstantSwap: Fast Customized Concept Swapping across Sharp Shape Differencescited by 34
  5. 2024SubjectDrive: Scaling Generative Data in Autonomous Driving via Subject Controlcited by 32
  6. 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 32
  7. 2024Open-Vocabulary Attention Maps with Token Optimization for Semantic Segmentation in Diffusion Modelscited by 29
  8. 2024SeaS: Few-shot Industrial Anomaly Image Generation with Separation and Sharing Fine-tuningcited by 27
  9. 2024DiffCut: Catalyzing Zero-Shot Semantic Segmentation with Diffusion Features and Recursive Normalized Cutcited by 26
  10. 2025Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentationcited by 23
  11. 2024DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecturecited by 22
  12. 2023LoCo: Locally Constrained Training-Free Layout-to-Image Synthesiscited by 21
38 more
  1. 2024Alfie: Democratising RGBA Image Generation With No cited by 20
  2. 2023SEGIC: Unleashing the Emergent Correspondence for In-Context Segmentationcited by 19
  3. 2023SegGen: Supercharging Segmentation Models with Text2Mask and Mask2Img Synthesiscited by 18
  4. 2024Frozen CLIP: A Strong Backbone for Weakly Supervised Semantic Segmentationcited by 15
  5. 2024Mask Factory: Towards High-quality Synthetic Data Generation for Dichotomous Image Segmentationcited by 15
  6. 2024Automated data processing and feature engineering for deep learning and big data applications: a surveycited by 14
  7. 2024GLASS: Guided Latent Slot Diffusion for Object-Centric Learningcited by 14
  8. 2023AI-Generated Images as Data Source: The Dawn of Synthetic Eracited by 13
  9. 2024Exploring Phrase-Level Grounding with Text-to-Image Diffusion Modelcited by 13
  10. 2025Unified Dense Prediction of Video Diffusioncited by 13
  11. 2024FlexEdit: Flexible and Controllable Diffusion-based Object-centric Image Editingcited by 12
  12. 2024Advancing Fine-Grained Classification by Structure and Subject Preserving Augmentationcited by 12
  13. 2025Attention (as Discrete-Time Markov) Chainscited by 11
  14. 2024iSeg: An Iterative Refinement-based Framework for Training-free Segmentationcited by 10
  15. 2024Emergent Visual Grounding in Large Multimodal Models Without Grounding Supervisioncited by 9
  16. 2025ReME: A Data-Centric Framework for Training-Free Open-Vocabulary Segmentationcited by 9
  17. 2024LLM-based Privacy Data Augmentation Guided by Knowledge Distillation with a Distribution Tutor for Medical Text Classificationcited by 8
  18. 2024Co-synthesis of Histopathology Nuclei Image-Label Pairs using a Context-Conditioned Joint Diffusion Modelcited by 8
  19. 2024Generative AI Enables Medical Image Segmentation in Ultra Low-Data Regimescited by 8
  20. 2024TextSSR: Diffusion-based Data Synthesis for Scene Text Recognitioncited by 8
  21. 2024AttenCraft: Attention-guided Disentanglement of Multiple Concepts for Text-to-Image Customizationcited by 6
  22. 2024SynFER: Towards Boosting Facial Expression Recognition with Synthetic Datacited by 6
  23. 2023ScribbleGen: Generative Data Augmentation Improves Scribble-supervised Semantic Segmentationcited by 5
  24. 2024FLAME Diffuser: Wildfire Image Synthesis using Mask Guided Diffusioncited by 5
  25. 2024WAS: Dataset and Methods for Artistic Text Segmentationcited by 5
  26. 2024Amodal Instance Segmentation with Diffusion Shape Prior Estimationcited by 5
  27. 2025A Unified Image-Dense Annotation Generation Model for Underwater Scenescited by 5
  28. 2025G4Seg: Generation for Inexact Segmentation Refinement with Diffusion Modelscited by 5
  29. 2025SeeDiff: Off-the-Shelf Seeded Mask Generation from Diffusion Modelscited by 5
  30. 2024Future-Proofing Class-Incremental Learningcited by 4
  31. 2024Robust Disaster Assessment from Aerial Imagery Using Text-to-Image Synthetic Datacited by 4
  32. 2024Training-Free Layout-to-Image Generation with Marginal Attention Constraintscited by 4
  33. 2025TRACE: Your Diffusion Model is Secretly an Instance Edge Detectorcited by 4
  34. 2025Aligning Generative Denoising with Discriminative Objectives Unleashes Diffusion for Visual Perceptioncited by 4
  35. 2025JoDiffusion: Jointly Diffusing Image with Pixel-Level Annotations for Semantic Segmentation Promotioncited by 4
  36. 2024On Good Practices for Task-Specific Distillation of Large Pretrained Visual Modelscited by 3
  37. 2024Weak-to-Strong Compositional Learning from Generative Models for Language-based Object Detectioncited by 3
  38. 2024BGDB: Bernoulli-Gaussian Decision Block with Improved Denoising Diffusion Probabilistic Modelscited by 3

50 most cited of 112 All 112 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Exploring Limits of Diffusion-Synthetic Training with Weakly Supervised Semantic Segmentation
  2. 2025Unsupervised Class Generation to Expand Semantic Segmentation Datasets
  3. 2023Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmenter
  4. 2026Making Training-Free Diffusion Segmentors Scale with the Generative Power
  5. 2024EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models
  6. 2026D3S2: Diffusion-Guided Dataset Distillation for Semantic Segmentation
  7. 2026From Diffusion to Rectified Flow: Rethinking Text-Based Segmentation
  8. 2023Open-vocabulary Object Segmentation with Diffusion Models