# Mask Factory: Towards High-quality Synthetic Data Generation for Dichotomous Image Segmentation

arXiv: https://arxiv.org/abs/2412.19080 · HTML: https://mlsanity.com/paper/2412.19080/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 41 references that are in the map.
- First posted on arXiv 26 Dec 2024.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [Enhanced-alignment Measure for Binary Foreground Map Evaluation](https://mlsanity.com/paper/1805.10421/) (2018)
- [SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation](https://mlsanity.com/paper/2004.01803/) (2020)
- [DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Models](https://mlsanity.com/paper/2308.06160/) (2023)
- [Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentation](https://mlsanity.com/paper/2309.14303/) (2023)
- [Bilateral Reference for High-Resolution Dichotomous Image Segmentation](https://mlsanity.com/paper/2401.03407/) (2024)
- [Highly Accurate Dichotomous Image Segmentation](https://mlsanity.com/paper/2203.03041/) (2022)
- [Towards Generalizable Tumor Synthesis](https://mlsanity.com/paper/2402.19470/) (2024)
- [FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Models](https://mlsanity.com/paper/2310.15160/) (2023)
- [Synthetic Benchmarks for Scientific Research in Explainable Machine Learning](https://mlsanity.com/paper/2106.12543/) (2021)
- [Generative Text-Guided 3D Vision-Language Pretraining for Unified Medical Image Segmentation](https://mlsanity.com/paper/2306.04811/) (2023)
- [SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation](https://mlsanity.com/paper/2403.16605/) (2024)
- [Explore the Power of Synthetic Data on Few-shot Object Detection](https://mlsanity.com/paper/2303.13221/) (2023)
- [BIMCV-R: A Landmark Dataset for 3D CT Text-Image Retrieval](https://mlsanity.com/paper/2403.15992/) (2024)
- [Multi-view Aggregation Network for Dichotomous Image Segmentation](https://mlsanity.com/paper/2404.07445/) (2024)
- [ExPT: Synthetic Pretraining for Few-Shot Experimental Design](https://mlsanity.com/paper/2310.19961/) (2023)
- [Rethinking Interactive Image Segmentation with Low Latency, High Quality, and Diverse Prompts](https://mlsanity.com/paper/2404.00741/) (2024)
- [Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2310.04148/) (2023)
- [Unite-Divide-Unite: Joint Boosting Trunk and Structure for High-accuracy Dichotomous Image Segmentation](https://mlsanity.com/paper/2307.14052/) (2023)
- [Learning Multiscale Consistency for Self-supervised Electron Microscopy Instance Segmentation](https://mlsanity.com/paper/2308.09917/) (2023)
- [Instructive Feature Enhancement for Dichotomous Medical Image Segmentation](https://mlsanity.com/paper/2306.03497/) (2023)

## Built on by (most cited first, 15 of 15)

- [HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrieval](https://mlsanity.com/paper/2512.02792/) (2025)
- [Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Tools](https://mlsanity.com/paper/2510.08480/) (2025)
- [CRISP-SAM2: SAM2 with Cross-Modal Interaction and Semantic Prompting for Multi-Organ Segmentation](https://mlsanity.com/paper/2506.23121/) (2025)
- [Conditional Latent Coding with Learnable Synthesized Reference for Deep Image Compression](https://mlsanity.com/paper/2502.09971/) (2025)
- [SED-MVS: Segmentation-Driven and Edge-Aligned Deformation Multi-View Stereo with Depth Restoration and Occlusion Constraint](https://mlsanity.com/paper/2503.13721/) (2025)
- [S3OD: Towards Generalizable Salient Object Detection with Synthetic Data](https://mlsanity.com/paper/2510.21605/) (2025)
- [DVP-MVS++: Synergize Depth-Normal-Edge and Harmonized Visibility Prior for Multi-View Stereo](https://mlsanity.com/paper/2506.13215/) (2025)
- [The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation](https://mlsanity.com/paper/2602.11318/) (2026)
- [Dual form Complementary Masking for Domain-Adaptive Image Segmentation](https://mlsanity.com/paper/2507.12008/) (2025)
- [LawDIS: Language-Window-based Controllable Dichotomous Image Segmentation](https://mlsanity.com/paper/2508.01152/) (2025)
- [OMUDA: Omni-level Masking for Unsupervised Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2512.12303/) (2025)
- [GenCAMO: Scene-Graph Contextual Decoupling for Environment-aware and Mask-free Camouflage Image-Dense Annotation Generation](https://mlsanity.com/paper/2601.01181/) (2026)
- [Toward Real-World High-Precision Image Matting and Segmentation](https://mlsanity.com/paper/2601.12080/) (2026)
- [PathAR: Structure-First Autoregressive Synthesis of Multimodal Pathology Images](https://mlsanity.com/paper/2606.01543/) (2026)
- [Beyond the Foreground: FOV-Aware Polyp Image Synthesis via Lesion-Guided Adaptive Mucosal Context Propagation](https://mlsanity.com/paper/2609.19966/) (2026)

## Abstract (arXiv)

Dichotomous Image Segmentation (DIS) tasks require highly precise annotations, and traditional dataset creation methods are labor intensive, costly, and require extensive domain expertise. Although using synthetic data for DIS is a promising solution to these challenges, current generative models and techniques struggle with the issues of scene deviations, noise-induced errors, and limited training sample variability. To address these issues, we introduce a novel approach, textbfourmodel, which provides a scalable solution for generating diverse and precise datasets, markedly reducing preparation time and costs. We first introduce a general mask editing method that combines rigid and non-rigid editing techniques to generate high-quality synthetic masks. Specially, rigid editing leverages geometric priors from diffusion models to achieve precise viewpoint transformations under zero-shot conditions, while non-rigid editing employs adversarial training and self-attention mechanisms for complex, topologically consistent modifications. Then, we generate pairs of high-resolution image and accurate segmentation mask using a multi-conditional control generation method. Finally, our experiments on the widely-used DIS5K dataset benchmark demonstrate superior performance in quality and efficiency compared to existing methods. The code is available at urlhttps://qian-hao-tian.github.io/MaskFactory/.