mlsanityData to 14 Oct 2026

Diffusion Features to Bridge Domain Gap for Semantic Segmentation

arXiv 2406.00777 ↗First posted 2 Jun 2024cs.CV cs.AIcited by 5

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Pre-trained diffusion models have demonstrated remarkable proficiency in synthesizing images across a wide range of scenarios with customizable prompts, indicating their effective capacity to capture universal features. Motivated by this, our study delves into the utilization of the implicit knowledge embedded within diffusion models to address challenges in cross-domain semantic segmentation. This paper investigates the approach that leverages the sampling and fusion techniques to harness the features of diffusion models efficiently. We propose DIffusion Feature Fusion (DIFF) as a backbone use for extracting and integrating effective semantic representations through the diffusion process. By leveraging the strength of text-to-image generation capability, we introduce a new training framework designed to implicitly learn posterior knowledge from it. Through rigorous evaluation in the contexts of domain generalization semantic segmentation, we establish that our methodology surpasses preceding approaches in mitigating discrepancies across distinct domains and attains the state-of-the-art (SOTA) benchmark.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 49 of its 61 references that are in the map. First posted on arXiv 2 Jun 2024.

Builds on

49 of 61 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmenter6 Sep
  2. 2023Diffusion-based Image Translation with Label Guidance for Domain Adaptive Semantic Segmentation23 Aug
  3. 2023Prompting Diffusion Representations for Cross-Domain Semantic Segmentation5 Jul
  4. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models8 Mar
  5. 2021Label-Efficient Semantic Segmentation with Diffusion Models6 Dec

Other references · 44

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2020Denoising Diffusion Implicit Modelscited by 9,781
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  9. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  10. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  11. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  12. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
32 more
  1. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  2. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  3. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  4. 2016Playing for Data: Ground Truth from Computer Gamescited by 818
  5. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  6. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  7. 2021Domain Generalization: A Surveycited by 764
  8. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 700
  9. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  10. 2023Emergent Correspondence from Image Diffusioncited by 446
  11. 2018Two at Once: Enhancing Learning and Generalization Capacities via IBN-Netcited by 364
  12. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 323
  13. 2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 300
  14. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 269
  15. 2019Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Datacited by 246
  16. 2020DACS: Domain Adaptation via Cross-domain Mixed Samplingcited by 226
  17. 2021RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whiteningcited by 215
  18. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 214
  19. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
  20. 2021FSDR: Frequency Space Domain Randomization for Domain Generalizationcited by 164
  21. 2022Diffusion-based Image Translation using Disentangled Style and Content Representationcited by 149
  22. 2022Uncovering the Disentanglement Capability in Text-to-Image Diffusion Modelscited by 116
  23. 2024Deconstructing Denoising Diffusion Models for Self-Supervised Learningcited by 91
  24. 2022Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentationcited by 89
  25. 2022Semantic-Aware Domain Generalized Segmentationcited by 79
  26. 2022Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentationcited by 70
  27. 2023Diffusion Models Beat GANs on Image Classificationcited by 58
  28. 2021Global and Local Texture Randomization for Synthetic-to-Real Semantic Segmentationcited by 57
  29. 2023Learning Content-enhanced Mask Transformer for Domain Generalized Urban-Scene Segmentationcited by 30
  30. 2023A Geometric Perspective on Diffusion Modelscited by 18
  31. 2023A Re-Parameterized Vision Transformer (ReVT) for Domain-Generalized Semantic Segmentationcited by 8
  32. 2023ALADIN-NST: Self-supervised disentangled representation learning of artistic style through Neural Style Transfercited by 3

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 4.

5 papers.

  1. 2024Game4Loc: A UAV Geo-Localization Benchmark from Game Datacited by 24
  2. 2025Generalized Diffusion Detector: Mining Robust Features from Diffusion Models for Domain-Generalized Detectioncited by 6
  3. 2025UPRE: Zero-Shot Domain Adaptation for Object Detection via Unified Prompt and Representation Enhancementcited by 4
  4. 2025Exploring Probabilistic Modeling Beyond Domain Generalization for Semantic Segmentationcited by 4
  5. 2025Latent Diffusion U-Net Representations Contain Positional Embeddings and Anomaliescited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models
  2. 2023Generalization by Adaptation: Diffusion-Based Domain Extension for Domain-Generalized Semantic Segmentation
  3. 2023Denoising Diffusion Semantic Segmentation with Mask Prior Modeling
  4. 2026Making Training-Free Diffusion Segmentors Scale with the Generative Power
  5. 2023DifFSS: Diffusion Model for Few-Shot Semantic Segmentation
  6. 2024Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation
  7. 2026Diffusion Model as a Generalist Segmentation Learner
  8. 2024High-Precision Dichotomous Image Segmentation via Probing Diffusion Capacity