# Diffusion Prism: Enhancing Diversity and Morphology Consistency in Mask-to-Image Diffusion

arXiv: https://arxiv.org/abs/2501.00944 · HTML: https://mlsanity.com/paper/2501.00944/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 1 Jan 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2305.16322/) (2023)
- [Brain Imaging Generation with Latent Diffusion Models](https://mlsanity.com/paper/2209.07162/) (2022)
- [Segment Anything in Medical Images](https://mlsanity.com/paper/2304.12306/) (2023)
- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [Denoising Diffusion Bridge Models](https://mlsanity.com/paper/2309.16948/) (2023)
- [MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformer](https://mlsanity.com/paper/2301.11798/) (2023)
- [DP-CGAN: Differentially Private Synthetic Data and Label Generation](https://mlsanity.com/paper/2001.09700/) (2020)
- [Transparent Image Layer Diffusion using Latent Transparency](https://mlsanity.com/paper/2402.17113/) (2024)
- [Custom-Edit: Text-Guided Image Editing with Customized Diffusion Models](https://mlsanity.com/paper/2305.15779/) (2023)
- [ArtFID: Quantitative Evaluation of Neural Style Transfer](https://mlsanity.com/paper/2207.12280/) (2022)
- [Exploiting the Signal-Leak Bias in Diffusion Models](https://mlsanity.com/paper/2309.15842/) (2023)
- [Face Generation and Editing with StyleGAN: A Survey](https://mlsanity.com/paper/2212.09102/) (2022)
- [MaskDiffusion: Boosting Text-to-Image Consistency with Conditional Mask](https://mlsanity.com/paper/2309.04399/) (2023)
- [OTRE: Where Optimal Transport Guided Unpaired Image-to-Image Translation Meets Regularization by Enhancing](https://mlsanity.com/paper/2302.03003/) (2023)
- [Fairness in Face Presentation Attack Detection](https://mlsanity.com/paper/2209.09035/) (2022)
- [MaskDiffusion: Exploiting Pre-trained Diffusion Models for Semantic Segmentation](https://mlsanity.com/paper/2403.11194/) (2024)
- [FLAME Diffuser: Wildfire Image Synthesis using Mask Guided Diffusion](https://mlsanity.com/paper/2403.03463/) (2024)
- [EMDS-6: Environmental Microorganism Image Dataset Sixth Version for Image Denoising, Segmentation, Feature Extraction, Classification and Detection Methods Evaluation](https://mlsanity.com/paper/2112.07111/) (2021)
- [Fast Key Points Detection and Matching for Tree-Structured Images](https://mlsanity.com/paper/2211.03242/) (2022)
- [RBAD: A Dataset and Benchmark for Retinal Vessels Branching Angle Detection](https://mlsanity.com/paper/2407.12271/) (2024)
- [Nano-Resolution Visual Identifiers Enable Secure Monitoring in Next-Generation Cyber-Physical Systems](https://mlsanity.com/paper/2211.08678/) (2022)
- [Enhancing Digital Hologram Reconstruction Using Reverse-Attention Loss for Untrained Physics-Driven Deep Learning Models with Uncertain Distance](https://mlsanity.com/paper/2403.12056/) (2024)

## Built on by (most cited first, 1 of 1)

- [AtomDiffuser: Time-Aware Degradation Modeling for Drift and Beam Damage in STEM Imaging](https://mlsanity.com/paper/2508.10359/) (2025)

## Abstract (arXiv)

The emergence of generative AI and controllable diffusion has made image-to-image synthesis increasingly practical and efficient. However, when input images exhibit low entropy and sparse, the inherent characteristics of diffusion models often result in limited diversity. This constraint significantly interferes with data augmentation. To address this, we propose Diffusion Prism, a training-free framework that efficiently transforms binary masks into realistic and diverse samples while preserving morphological features. We explored that a small amount of artificial noise will significantly assist the image-denoising process. To prove this novel mask-to-image concept, we use nano-dendritic patterns as an example to demonstrate the merit of our method compared to existing controllable diffusion models. Furthermore, we extend the proposed framework to other biological patterns, highlighting its potential applications across various fields.