mlsanityData to 7 Oct 2026

Repurposing GANs for One-shot Semantic Part Segmentation

arXiv 2103.04379 ↗First posted 7 Mar 2021cs.CV cs.LGcited by 88

Not on the map. Why?

Cited by 29 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

While GANs have shown success in realistic image generation, the idea of using GANs for other tasks unrelated to synthesis is underexplored. Do GANs learn meaningful structural parts of objects during their attempt to reproduce those objects? In this work, we test this hypothesis and propose a simple and effective approach based on GANs for semantic part segmentation that requires as few as one label example along with an unlabeled dataset. Our key idea is to leverage a trained GAN to extract pixel-wise representation from the input image and use it as feature vectors for a segmentation network. Our experiments demonstrate that GANs representation is "readily discriminative" and produces surprisingly good results that are comparable to those from supervised baselines trained with significantly more labels. We believe this novel repurposing of GANs underlies a new class of unsupervised representation learning that is applicable to many other tasks. More results are available at https://repurposegans.github.io/.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). Builds on 61 of its 83 references that are in the map. First posted on arXiv 7 Mar 2021.

Builds on

61 of 83 references.

References · 61

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  7. 2013Auto-Encoding Variational Bayescited by 3,980
  8. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  10. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  11. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  12. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
38 more
  1. 2014Conditional Generative Adversarial Netscited by 2,152
  2. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  3. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  4. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  5. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  6. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  7. 2016Pixel Recurrent Neural Networkscited by 814
  8. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  9. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  10. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  11. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  12. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  13. 2015Holistically-Nested Edge Detectioncited by 590
  14. 2019PANet: Few-Shot Image Semantic Segmentation with Prototype Alignmentcited by 449
  15. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
  16. 2017One-Shot Learning for Semantic Segmentationcited by 337
  17. 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
  18. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 299
  19. 2020Unsupervised Discovery of Interpretable Directions in the GAN Latent Spacecited by 288
  20. 2017AttGAN: Facial Attribute Editing by Only Changing What You Wantcited by 241
  21. 2019Large Scale Adversarial Representation Learningcited by 238
  22. 2016One-Shot Video Object Segmentationcited by 233
  23. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 230
  24. 2013OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networkscited by 224
  25. 2020Editing in Style: Uncovering the Local Semantics of GANscited by 183
  26. 2017Colorization as a Proxy Task for Visual Understandingcited by 175
  27. 2018SG-One: Similarity Guidance Network for One-Shot Semantic Segmentationcited by 169
  28. 2018Isolating Sources of Disentanglement in Variational Autoencoderscited by 167
  29. 2020Part-aware Prototype Network for Few-shot Semantic Segmentationcited by 162
  30. 2015Instance-aware Semantic Segmentation via Multi-task Network Cascadescited by 120
  31. 2020Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimationcited by 120
  32. 2018Unsupervised Learning of Object Landmarks through Conditional Image Generationcited by 113
  33. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  34. 2018Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoningcited by 79
  35. 2018TernausNet: U-Net with VGG11 Encoder Pre-Trained on ImageNet for Image Segmentationcited by 76
  36. 2017Neural Face Editing with Intrinsic Image Disentanglingcited by 70
  37. 2017Unsupervised learning of object landmarks by factorized spatial embeddingscited by 64
  38. 2014Learning Deep Representation for Face Alignment with Auxiliary Attributescited by 32

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 16; 2023: 32; 2024: 23; 2025: 6; 2026: 2.

88 papers.

Sort:

By area:

  1. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  2. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  3. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
  4. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 326
  5. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 218
  6. 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
  7. 2021Generative Models as a Data Source for Multiview Representation Learningcited by 104
  8. 2023Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentationcited by 100
  9. 2022BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotationscited by 99
  10. 2021Learning to See by Looking at Noisecited by 92
  11. 2021CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directionscited by 78
  12. 2021Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networkscited by 68
38 more
  1. 2023Real-Time Radiance Fields for Single-Image Portrait View Synthesiscited by 64
  2. 2021GAN-Supervised Dense Visual Alignmentcited by 61
  3. 2021Unsupervised Part Discovery from Contrastive Reconstructioncited by 57
  4. 2021Ensembling with Deep Generative Viewscited by 53
  5. 2024EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Modelscited by 42
  6. 2023Exploiting Diffusion Prior for Generalizable Dense Predictioncited by 37
  7. 2024EditWorld: Simulating World Dynamics for Instruction-Following Image Editingcited by 37
  8. 2022State-of-the-Art in the Architecture, Methods and Applications of StyleGANcited by 34
  9. 2023SLiMe: Segment Like Mecited by 34
  10. 2024Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Modelscited by 32
  11. 2023LD-ZNet: A Latent Diffusion Approach for Text-Based Image Segmentationcited by 29
  12. 2022User-Controllable Latent Transformer for StyleGAN Image Layout Editingcited by 28
  13. 2023PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentationcited by 19
  14. 2023CHORUS: Learning Canonicalized 3D Human-Object Spatial Relations from Unbounded Synthesized Imagescited by 19
  15. 2023Few Shot Part Segmentation Reveals Compositional Logic for Industrial Anomaly Detectioncited by 19
  16. 2024Synth2: Boosting Visual-Language Models with Synthetic Captions and Image Embeddingscited by 18
  17. 2022Synthetic Data Supervised Salient Object Detectioncited by 15
  18. 2023Towards Open-World Segmentation of Partscited by 15
  19. 2025DC-SAM: In-Context Segment Anything in Images and Videos via Dual Consistencycited by 15
  20. 2021GANORCON: Are Generative Models Useful for Few-shot Segmentation?cited by 13
  21. 2023AI-Generated Images as Data Source: The Dawn of Synthetic Eracited by 13
  22. 2024Exploring Phrase-Level Grounding with Text-to-Image Diffusion Modelcited by 13
  23. 2024Explore In-Context Segmentation via Latent Diffusion Modelscited by 12
  24. 2024MaskDiffusion: Exploiting Pre-trained Diffusion Models for Semantic Segmentationcited by 11
  25. 2022Improving Few-Shot Part Segmentation using Coarse Supervisioncited by 10
  26. 2023Progressive Learning of 3D Reconstruction Network from 2D GAN Datacited by 10
  27. 2023DemoCaricature: Democratising Caricature Generation with a Rough Sketchcited by 10
  28. 2024Concept Replacer: Replacing Sensitive Concepts in Diffusion Models via Precision Localizationcited by 9
  29. 2022Correspondence Distillation from NeRF-based GANcited by 8
  30. 2023Controllable Visual-Tactile Synthesiscited by 8
  31. 2023GeNIe: Generative Hard Negative Images Through Diffusioncited by 8
  32. 2023Instance-Conditioned GAN Data Augmentation for Representation Learningcited by 7
  33. 2024SMITE: Segment Me In TimEcited by 7
  34. 2023Few-shot Geometry-Aware Keypoint Localizationcited by 6
  35. 2024Few-Shot Anomaly Detection via Category-Agnostic Registration Learningcited by 6
  36. 2022Extracting Semantic Knowledge from GANs with Unsupervised Learningcited by 5
  37. 2023PARTICLE: Part Discovery and Contrastive Learning for Fine-grained Recognitioncited by 5
  38. 2024Synthetic to Authentic: Transferring Realism to 3D Face Renderings for Boosting Face Recognitioncited by 5

50 most cited of 88 All 88 (JSON)