mlsanityData to 14 Oct 2026

PaintSeg: Training-free Segmentation via Painting

arXiv 2305.19406 ↗First posted 30 May 2023cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

The paper introduces PaintSeg, a new unsupervised method for segmenting objects without any training. We propose an adversarial masked contrastive painting (AMCP) process, which creates a contrast between the original image and a painted image in which a masked area is painted using off-the-shelf generative models. During the painting process, inpainting and outpainting are alternated, with the former masking the foreground and filling in the background, and the latter masking the background while recovering the missing part of the foreground object. Inpainting and outpainting, also referred to as I-step and O-step, allow our method to gradually advance the target segmentation mask toward the ground truth without supervision or training. PaintSeg can be configured to work with a variety of prompts, e.g. coarse masks, boxes, scribbles, and points. Our experimental results demonstrate that PaintSeg outperforms existing approaches in coarse mask-prompt, box-prompt, and point-prompt segmentation tasks, providing a training-free solution suitable for unsupervised segmentation.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 53 of its 75 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

53 of 75 references.

References · 53

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2023Segment Anythingcited by 8,234
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  8. 2017Mask R-CNNcited by 7,277
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  12. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
38 more
  1. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  2. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  3. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  4. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 431
  5. 2016Segmentation from Natural Language Expressionscited by 257
  6. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 230
  7. 2021Associating Objects with Transformers for Video Object Segmentationcited by 226
  8. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 212
  9. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  10. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 187
  11. 2022Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cutcited by 167
  12. 2021Reviving Iterative Training with Mask Guidance for Interactive Segmentationcited by 163
  13. 2016Deep Interactive Object Selectioncited by 160
  14. 2022Audio-Visual Segmentationcited by 151
  15. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  16. 2020Collaborative Video Object Segmentation by Foreground-Background Integrationcited by 142
  17. 2022FreeSOLO: Learning to Segment Objects without Annotationscited by 102
  18. 2020Video Object Segmentation with Adaptive Feature Bank and Uncertain-Region Refinementcited by 101
  19. 2015The Fast Bilateral Solvercited by 87
  20. 2021Efficient Regional Memory Network for Video Object Segmentationcited by 86
  21. 2022Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encodingcited by 83
  22. 2022Generating High Fidelity Data from Low-density Regions using Diffusion Modelscited by 78
  23. 2019Unsupervised Object Segmentation by Redrawingcited by 74
  24. 2019Unsupervised Moving Object Detection via Contextual Information Separationcited by 72
  25. 2022Unsupervised Object Localization: Observing the Background to Discover Objectscited by 66
  26. 2019Emergence of Object Segmentation in Perturbed Generative Modelscited by 57
  27. 2021Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Searchcited by 57
  28. 2022Unsupervised Salient Object Detection with Spectral Cluster Votingcited by 56
  29. 2020Object Segmentation Without Labels with Large-Scale Generative Modelscited by 52
  30. 2022Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentationcited by 50
  31. 2021Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Modelscited by 44
  32. 2021EdgeFlow: Achieving Practical Interactive Segmentation with Edge-Guided Flowcited by 40
  33. 2017LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generationcited by 36
  34. 2023TarViS: A Unified Approach for Target-based Video Segmentationcited by 33
  35. 2019OneGAN: Simultaneous Unsupervised Learning of Conditional Image Generation, Foreground Segmentation, and Fine-Grained Clusteringcited by 28
  36. 2022Multi-Attention Network for Compressed Video Referring Object Segmentationcited by 28
  37. 2021GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generationcited by 19
  38. 2022Modeling Motion with Multi-Modal Features for Text-Based Video Segmentationcited by 18

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 3; 2025: 2.

7 papers.

  1. 2023Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Surveycited by 16
  2. 2024Repurposing Stable Diffusion Attention for Training-Free Unsupervised Interactive Segmentationcited by 9
  3. 2024Part-aware Prompted Segment Anything Model for Adaptive Segmentationcited by 5
  4. 2023Completing Visual Objects via Bridging Generation and Segmentationcited by 2
  5. 2024Artificial Intelligence for Geometry-Based Feature Extraction, Analysis and Synthesis in Artistic Images: A Surveycited by 1
  6. 2025M2N2V2: Multi-Modal Unsupervised and Training-free Interactive Segmentationcited by 1
  7. 2025Weakly Supervised Object Segmentation by Background Conditional Divergencecited by 0