Principal Component Clustering for Semantic Segmentation in Synthetic Data Generation
On the map: Diffusion models → Medical imaging and segmentation
Abstract (arXiv)
This technical report outlines our method for generating a synthetic dataset for semantic segmentation using a latent diffusion model. Our approach eliminates the need for additional models specifically trained on segmentation data and is part of our submission to the CVPR 2024 workshop challenge, entitled CVPR 2024 workshop challenge "SyntaGen Harnessing Generative Models for Synthetic Visual Datasets". Our methodology uses self-attentions to facilitate a novel head-wise semantic information condensation, thereby enabling the direct acquisition of class-agnostic image segmentation from the Stable Diffusion latents. Furthermore, we employ non-prompt-influencing cross-attentions from text to pixel, thus facilitating the classification of the previously generated masks. Finally, we propose a mask refinement step by using only the output image by Stable Diffusion.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 5 of its 5 references that are in the map. First posted on arXiv 25 Jun 2024.
Builds on
5 of 5 references.
Closest ancestors · 1
Same direction, newest first
- 2024Open-Vocabulary Attention Maps with Token Optimization for Semantic Segmentation in Diffusion Models21 Mar
Other references · 4
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
- 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 233
- 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Exploring Limits of Diffusion-Synthetic Training with Weakly Supervised Semantic Segmentation
- 2024Free-Mask: A Novel Paradigm of Integration Between the Segmentation Diffusion Model and Image Editing
- 2023DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Models
- 2023Masked Diffusion as Self-supervised Representation Learner
- 2025SeeDiff: Off-the-Shelf Seeded Mask Generation from Diffusion Models
- 2026Making Training-Free Diffusion Segmentors Scale with the Generative Power
- 2026What Makes Synthetic Data Effective in Image Segmentation
- 2023Denoising Diffusion Semantic Segmentation with Mask Prior Modeling