mlsanityData to 14 Oct 2026

FreeMask: Synthetic Images with Dense Annotations Make Stronger Segmentation Models

arXiv 2310.15160 ↗First posted 23 Oct 2023cs.CVcited by 61

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Semantic segmentation has witnessed tremendous progress due to the proposal of various advanced network architectures. However, they are extremely hungry for delicate annotations to train, and the acquisition is laborious and unaffordable. Therefore, we present FreeMask in this work, which resorts to synthetic images from generative models to ease the burden of both data collection and annotation procedures. Concretely, we first synthesize abundant training images conditioned on the semantic masks provided by realistic datasets. This yields extra well-aligned image-mask training pairs for semantic segmentation models. We surprisingly observe that, solely trained with synthetic images, we already achieve comparable performance with real ones (e.g., 48.3 vs. 48.5 mIoU on ADE20K, and 49.3 vs. 50.5 on COCO-Stuff). Then, we investigate the role of synthetic images by joint training with real images, or pre-training for real images. Meantime, we design a robust filtering principle to suppress incorrectly synthesized regions. In addition, we propose to inequally treat different semantic masks to prioritize those harder ones and sample more corresponding synthetic images for them. As a result, either jointly trained or pre-trained with our filtered and re-sampled synthesized images, segmentation models can be greatly enhanced, e.g., from 48.7 to 52.0 on ADE20K. Code is available at https://github.com/LiheYoung/FreeMask.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). Builds on 65 of its 75 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

65 of 75 references.

References · 65

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  10. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
38 more
  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2022Generative Adversarial Networkscited by 5,627
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  7. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  8. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  10. 2016Pyramid Scene Parsing Networkcited by 2,479
  11. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  12. 2017Non-local Neural Networkscited by 2,293
  13. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  14. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  15. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  16. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  17. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  18. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  19. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  20. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  21. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  22. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  23. 2016Playing for Data: Ground Truth from Computer Gamescited by 818
  24. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  25. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 718
  26. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 695
  27. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 579
  28. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  29. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 492
  30. 2016RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentationcited by 465
  31. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 327
  32. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  33. 2023Effective Data Augmentation With Diffusion Modelscited by 283
  34. 2017Data Augmentation Generative Adversarial Networkscited by 278
  35. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  36. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  37. 2017Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Networkcited by 208
  38. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 172

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 23; 2025: 22; 2026: 12.

61 papers.

Sort:

By area:

  1. 2023DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Controlcited by 45
  2. 2024Large-Scale 3D Medical Image Pre-training with Geometric Context Priorscited by 36
  3. 2025Triad: Vision Foundation Model for 3D Magnetic Resonance Imagingcited by 24
  4. 2024PLACE: Adaptive Layout-Semantic Fusion for Semantic Image Synthesiscited by 20
  5. 2023SegGen: Supercharging Segmentation Models with Text2Mask and Mask2Img Synthesiscited by 18
  6. 2024ODGEN: Domain-specific Object Detection Data Generation with Diffusion Modelscited by 17
  7. 2024Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentationcited by 16
  8. 2024DiverGen: Improving Instance Segmentation by Learning Wider Data Distribution with More Diverse Generative Datacited by 15
  9. 2024FreeTumor: Advance Tumor Segmentation via Large-Scale Tumor Synthesiscited by 15
  10. 2024FairDiff: Fair Segmentation with Point-Image Diffusioncited by 15
  11. 2024Image Understanding Makes for A Good Tokenizer for Image Generationcited by 15
  12. 2024Mask Factory: Towards High-quality Synthetic Data Generation for Dichotomous Image Segmentationcited by 15
38 more
  1. 2023Open-Vocabulary Camouflaged Object Segmentationcited by 14
  2. 2024Generative Enhancement for 3D Medical Imagescited by 14
  3. 2024DiffuMatting: Synthesizing Arbitrary Objects with Matting-level Annotationcited by 13
  4. 2024Generative Active Learning for Long-tailed Instance Segmentationcited by 10
  5. 2024A Simple Background Augmentation Method for Object Detection with Diffusion Modelcited by 9
  6. 2024Repurposing Stable Diffusion Attention for Training-Free Unsupervised Interactive Segmentationcited by 9
  7. 2024SCP-Diff: Spatial-Categorical Joint Prior for Diffusion Based Semantic Image Synthesiscited by 7
  8. 2023ScribbleGen: Generative Data Augmentation Improves Scribble-supervised Semantic Segmentationcited by 5
  9. 2025Taste More, Taste Better: Diverse Data and Strong Model Boost Semi-Supervised Crowd Countingcited by 5
  10. 2025A Unified Image-Dense Annotation Generation Model for Underwater Scenescited by 5
  11. 2024Robust Disaster Assessment from Aerial Imagery Using Text-to-Image Synthetic Datacited by 4
  12. 2024Single Image, Any Face: Generalisable 3D Face Generationcited by 4
  13. 2024Ctrl-GenAug: Controllable Generative Augmentation for Medical Sequence Classificationcited by 4
  14. 2025Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMscited by 4
  15. 2025JoDiffusion: Jointly Diffusing Image with Pixel-Level Annotations for Semantic Segmentation Promotioncited by 4
  16. 2024On Good Practices for Task-Specific Distillation of Large Pretrained Visual Modelscited by 3
  17. 2024SynDiff-AD: Improving Semantic Segmentation and End-to-End Autonomous Driving with Synthetic Data from Latent Diffusion Modelscited by 3
  18. 2025PathoPainter: Augmenting Histopathology Segmentation via Tumor-aware Inpaintingcited by 3
  19. 2025DreamMask: Boosting Open-vocabulary Panoptic Segmentation with Synthetic Datacited by 2
  20. 2025DIVE: Inverting Conditional Diffusion Models for Discriminative Taskscited by 2
  21. 2025Improving Knowledge Distillation Under Unknown Covariate Shift Through Confidence-Guided Data Augmentationcited by 2
  22. 2025NRSeg: Noise-Resilient Learning for BEV Semantic Segmentation via Driving World Modelscited by 2
  23. 2025Task-Oriented Data Synthesis and Control-Rectify Sampling for Remote Sensing Semantic Segmentationcited by 2
  24. 2026Benchmarking Semantic Segmentation Models via Appearance and Geometry Attribute Editingcited by 2
  25. 2024Boosting Few-Shot Detection with Large Language Models and Layout-to-Image Synthesiscited by 1
  26. 2024Paint Outside the Box: Synthesizing and Selecting Training Data for Visual Groundingcited by 1
  27. 2025PoseSyn: Synthesizing Diverse 3D Pose Data from In-the-Wild 2D Datacited by 1
  28. 2025M2N2V2: Multi-Modal Unsupervised and Training-free Interactive Segmentationcited by 1
  29. 2025SynRES: Towards Referring Expression Segmentation in the Wild via Synthetic Datacited by 1
  30. 2025Subtyping Breast Lesions via Generative Augmentation based Long-tailed Recognition in Ultrasoundcited by 1
  31. 2025InstaDA: Augmenting Instance Segmentation Data with Dual-Agent Systemcited by 1
  32. 2025Beyond the Seen: Bounded Distribution Estimation for Open-Vocabulary Learningcited by 1
  33. 2026Leveraging Image Generators to Address Data Scarcity: The Gen4Regen Dataset for Forest Regeneration Mappingcited by 1
  34. 2026FrozenDrive: Zero-Shot Text-Guided Driving Scene Generation and Data Augmentation with Parameter-Free Frozen Diffusion Modelcited by 1
  35. 2026TMI: Text-to-Image Meets Image-to-Image for Complementary Data Synthesis to Boost Long-Tailed Instance Segmentationcited by 1
  36. 2024How Vision-Language Tasks Benefit from Large Pre-trained Models: A Surveycited by 0
  37. 2025CA-LoRA: Concept-Aware LoRA for Domain-Aligned Segmentation Dataset Generationcited by 0
  38. 2025Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoderscited by 0

50 most cited of 61 All 61 (JSON)