mlsanityData to 14 Oct 2026

What Makes Synthetic Data Effective in Image Segmentation

arXiv 2605.19289 ↗First posted 19 May 2026cs.CVcited by 0

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Driven by rapid advances in large-scale generative models, synthetic data has emerged as a promising solution for visual understanding. While modern diffusion models achieve remarkable photorealistic image synthesis, their potential in complex visual segmentation tasks remains underexplored. In this work, we conduct a systematic analysis of synthetic images from state-of-the-art diffusion models to uncover the factors governing their utility. In particular, synthetic images characterized by dense scene composition and fine instance fidelity demonstrate distinctive benefits, yielding significantly more discriminative spatial representations. Building on these insights, we propose SENSE, a unified framework that leverages flexible and scalable synthetic data to substantially enhance segmentation performance. Notably, SENSE is model-agnostic, compatible with diverse architectures (e.g., DPT and Mask2Former), and scales effectively across models with varying parameter capacities. Extensive experiments on Cityscapes, COCO, and ADE20K validate the effectiveness and generalization capability of our approach. Code is available at https://github.com/zhang0jhon/SENSE.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 63 of its 75 references that are in the map. First posted on arXiv 19 May 2026.

Builds on

63 of 75 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025JoDiffusion: Jointly Diffusing Image with Pixel-Level Annotations for Semantic Segmentation Promotion15 Dec
  2. 2023MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation22 Sep
  3. 2021Label-Efficient Semantic Segmentation with Diffusion Models6 Dec

Other references · 60

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  9. 2020End-to-End Object Detection with Transformerscited by 7,414
  10. 2022Scalable Diffusion Models with Transformerscited by 6,054
  11. 2022Flow Matching for Generative Modelingcited by 5,702
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
38 more
  1. 2017Neural Discrete Representation Learningcited by 4,712
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  6. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  10. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  11. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  12. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  13. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  14. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  15. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  16. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  17. 2025Gemma 3 Technical Reportcited by 1,824
  18. 2025DINOv3cited by 1,414
  19. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  20. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  21. 2024Phi-4 Technical Reportcited by 711
  22. 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
  23. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 327
  24. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  25. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  26. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 259
  27. 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Modelscited by 179
  28. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 172
  29. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  30. 2021Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalizationcited by 141
  31. 2023DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Modelscited by 123
  32. 2023Machine Learning for Synthetic Data Generation: A Reviewcited by 120
  33. 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
  34. 2023Scaling Laws of Synthetic Images for Model Training ... for Nowcited by 108
  35. 2024Best Practices and Lessons Learned on Synthetic Datacited by 106
  36. 2023GenAug: Retargeting behaviors to unseen situations via Generative Augmentationcited by 104
  37. 2022BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotationscited by 99
  38. 2025Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent Diffusion Modelscited by 65

50 most cited of 60 All 60 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026R&D: Balancing Reliability and Diversity in Synthetic Data Augmentation for Semantic Segmentation
  2. 2023Reliability in Semantic Segmentation: Can We Use Synthetic Data?
  3. 2025A Framework for Low-Effort Training Data Generation for Urban Semantic Segmentation
  4. 2024Free-Mask: A Novel Paradigm of Integration Between the Segmentation Diffusion Model and Image Editing
  5. 2023Exploring Limits of Diffusion-Synthetic Training with Weakly Supervised Semantic Segmentation
  6. 2024Principal Component Clustering for Semantic Segmentation in Synthetic Data Generation
  7. 2025Synthetic Object Compositions for Scalable and Accurate Learning in Detection, Segmentation, and Grounding
  8. 2026When Pretty Isn't Useful: Investigating Why Modern Text-to-Image Models Fail as Reliable Training Data Generators