mlsanityData to 14 Oct 2026

Unsupervised Part Discovery from Contrastive Reconstruction

arXiv 2111.06349 ↗First posted 11 Nov 2021cs.CV cs.LGcited by 57

Not on the map. Why?

Abstract (arXiv)

The goal of self-supervised visual representation learning is to learn strong, transferable image representations, with the majority of research focusing on object or scene level. On the other hand, representation learning at part level has received significantly less attention. In this paper, we propose an unsupervised approach to object part discovery and segmentation and make three contributions. First, we construct a proxy task through a set of objectives that encourages the model to learn a meaningful decomposition of the image into its parts. Secondly, prior work argues for reconstructing or clustering pre-computed features as a proxy to parts; we show empirically that this alone is unlikely to find meaningful parts; mainly because of their low resolution and the tendency of classification networks to spatially smear out information. We suggest that image reconstruction at the level of pixels can alleviate this problem, acting as a complementary cue. Lastly, we show that the standard evaluation based on keypoint regression does not correlate well with segmentation quality and thus introduce different metrics, NMI and ARI, that better characterize the decomposition of objects into parts. Our method yields semantic parts which are consistent across fine-grained but visually distinct categories, outperforming the state of the art on three benchmark datasets. Code is available at the project page: https://www.robots.ox.ac.uk/~vgg/research/unsup-parts/.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Builds on 68 of its 98 references that are in the map. First posted on arXiv 11 Nov 2021.

Builds on

68 of 98 references.

References · 68

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  10. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  11. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  12. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
38 more
  1. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  2. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  3. 2016Colorful Image Colorizationcited by 1,114
  4. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  5. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
  6. 2020Object-Centric Learning with Slot Attentioncited by 876
  7. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  8. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  9. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  10. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  11. 2020Hard Negative Mixing for Contrastive Learningcited by 402
  12. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  13. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
  14. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  15. 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
  16. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 232
  17. 2015Bilinear CNNs for Fine-grained Visual Recognitioncited by 228
  18. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  19. 2019Contrastive Learning of Structured World Modelscited by 209
  20. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  21. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  22. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  23. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  24. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  25. 2014Part-based R-CNNs for Fine-grained Category Detectioncited by 109
  26. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
  27. 2018Learning to Navigate for Fine-grained Classificationcited by 103
  28. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  29. 2019Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognitioncited by 95
  30. 2021Repurposing GANs for One-shot Semantic Part Segmentationcited by 88
  31. 2019Stacked Capsule Autoencoderscited by 82
  32. 2020Interpretable and Accurate Fine-grained Recognition via Region Groupingcited by 75
  33. 2018Unsupervised Discovery of Object Landmarks as Structural Representationscited by 73
  34. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  35. 2014The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classificationcited by 67
  36. 2019Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification from the Bottom Upcited by 66
  37. 2017Unsupervised learning of object landmarks by factorized spatial embeddingscited by 64
  38. 2018Deep Feature Factorization For Concept Discoverycited by 64

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 18; 2023: 16; 2024: 13; 2025: 7; 2026: 1.

57 papers.

Sort:

By area:

  1. 2021Deep ViT Features as Dense Visual Descriptorscited by 330
  2. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 196
  3. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  4. 2022SemMAE: Semantic-Guided Masking for Learning Masked Autoencoderscited by 114
  5. 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
  6. 2022LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discoverycited by 66
  7. 2022Improving Object-centric Learning with Query Optimizationcited by 60
  8. 2021Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objectscited by 50
  9. 2023FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Modelscited by 47
  10. 2022One-Shot Transfer of Affordance Regions? AffCorrs!cited by 31
  11. 2022Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemblecited by 31
  12. 2023Neural Congealing: Aligning Images to a Joint Semantic Atlascited by 31
38 more
  1. 2023ASIC: Aligning Sparse in-the-wild Image Collectionscited by 27
  2. 2022DeepCut: Unsupervised Segmentation using Graph Neural Networks Clusteringcited by 25
  3. 2023PDiscoNet: Semantically consistent part discovery for fine-grained recognitioncited by 21
  4. 2022Unsupervised Multi-object Segmentation by Predicting Probable Motion Patternscited by 19
  5. 2023PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentationcited by 19
  6. 2022AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypointscited by 17
  7. 2023Unsupervised Keypoints from Pretrained Diffusion Modelscited by 15
  8. 2025Adaptive Part Learning for Fine-Grained Generalized Category Discovery: A Plug-and-Play Enhancementcited by 13
  9. 2023Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attentioncited by 12
  10. 2024SlotLifter: Slot-guided Feature Lifting for Learning Object-centric Radiance Fieldscited by 12
  11. 2024Improving Visual Recognition with Hyperbolical Visual Hierarchy Mappingcited by 11
  12. 2024PDiscoFormer: Relaxing Part Discovery Constraints with Vision Transformerscited by 9
  13. 2023Cycle Consistency Driven Object Discoverycited by 7
  14. 2025GECO: Geometrically Consistent Embedding with Lightspeed Inferencecited by 6
  15. 2023PARTICLE: Part Discovery and Contrastive Learning for Fine-grained Recognitioncited by 5
  16. 2026PCA-Seg: Revisiting Cost Aggregation for Open-Vocabulary Semantic and Part Segmentationcited by 5
  17. 2022Leveraging GAN Priors for Few-Shot Part Segmentationcited by 4
  18. 2023Unsupervised Discovery of 3D Hierarchical Structure with Generative Diffusion Featurescited by 4
  19. 2023Learning Part Segmentation from Synthetic Animalscited by 4
  20. 2024Pose Prior Learner: Unsupervised Categorical Prior Learning for Pose Estimationcited by 4
  21. 2024Data-Efficient Inference of Neural Fluid Fields via SciML Foundation Modelcited by 4
  22. 2024CALICO: Part-Focused Semantic Co-Segmentation with Large Vision-Language Modelscited by 4
  23. 2025Fine-Grained Image-Text Correspondence with Cost Aggregation for Open-Vocabulary Part Segmentationcited by 4
  24. 2023Graphics Capsule: Learning Hierarchical 3D Face Representations from 2D Imagescited by 3
  25. 2024Understanding Multi-Granularity for Open-Vocabulary Part Segmentationcited by 3
  26. 2022Exemplar Learning for Medical Image Segmentationcited by 2
  27. 2022ILSGAN: Independent Layer Synthesis for Unsupervised Foreground-Background Segmentationcited by 2
  28. 2022Parsing Objects at a Finer Granularity: A Surveycited by 2
  29. 2024Composing Parts for Expressive Object Generationcited by 2
  30. 2024Unsupervised Part Discovery via Dual Representation Alignmentcited by 2
  31. 2024Disentangling Visual Priors: Unsupervised Learning of Scene Interpretations with Compositional Autoencodercited by 2
  32. 2025PartCo: Part-Level Correspondence Priors Enhance Category Discoverycited by 2
  33. 2022Relational Proxies: Emergent Relationships as Fine-Grained Discriminatorscited by 1
  34. 2023Bridging Neural and Symbolic Representations with Transitional Dictionary Learningcited by 1
  35. 2023Self-supervised Scene Text Segmentation with Object-centric Layered Representations Augmented by Text Regionscited by 1
  36. 2024Bidirectional Multi-Step Domain Generalization for Visible-Infrared Person Re-Identificationcited by 1
  37. 2025Unsupervised Part Discovery via Descriptor-Based Masked Image Restoration with Optimized Constraintscited by 1
  38. 2022LatentKeypointGAN: Controlling Images via Latent Keypoints -- Extended Abstractcited by 0

50 most cited of 57 All 57 (JSON)