mlsanityData to 7 Oct 2026

CP2: Copy-Paste Contrastive Pretraining for Semantic Segmentation

arXiv 2203.11709 ↗First posted 22 Mar 2022cs.CV cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

Recent advances in self-supervised contrastive learning yield good image-level representation, which favors classification tasks but usually neglects pixel-level detailed information, leading to unsatisfactory transfer performance to dense prediction tasks such as semantic segmentation. In this work, we propose a pixel-wise contrastive learning method called CP2 (Copy-Paste Contrastive Pretraining), which facilitates both image- and pixel-level representation learning and therefore is more suitable for downstream dense prediction tasks. In detail, we copy-paste a random crop from an image (the foreground) onto different background images and pretrain a semantic segmentation model with the objective of 1) distinguishing the foreground pixels from the background pixels, and 2) identifying the composed images that share the same foreground.Experiments show the strong performance of CP2 in downstream semantic segmentation: By finetuning CP2 pretrained models on PASCAL VOC 2012, we obtain 78.6% mIoU with a ResNet-50 and 79.5% with a ViT-S.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 46 of its 55 references that are in the map. First posted on arXiv 22 Mar 2022.

Builds on

46 of 55 references.

References · 46

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  10. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  11. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  12. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
34 more
  1. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  2. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  3. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  4. 2020Supervised Contrastive Learningcited by 2,803
  5. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  6. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  7. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  8. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  9. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  10. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  11. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  12. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  13. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  14. 2016Colorful Image Colorizationcited by 1,113
  15. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  16. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  17. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  18. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  19. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  20. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  21. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 595
  22. 2016Adversarial Feature Learningcited by 350
  23. 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
  24. 2019Large Scale Adversarial Representation Learningcited by 238
  25. 2019Unsupervised Embedding Learning via Invariant and Spreading Instance Featurecited by 231
  26. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  27. 2016Adversarially Learned Inferencecited by 216
  28. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  29. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  30. 2021Instance Localization for Self-supervised Detection Pretrainingcited by 99
  31. 2020Contrastive Learning for Label-Efficient Semantic Segmentationcited by 93
  32. 2018Iterative Reorganization with Weak Spatial Constraints: Solving Arbitrary Jigsaw Puzzles for Unsupervised Representation Learningcited by 51
  33. 2020CO2: Consistent Contrast for Unsupervised Visual Representation Learningcited by 38
  34. 2019Learning about an exponential amount of conditional distributionscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 12; 2024: 4; 2025: 2; 2026: 3.

26 papers.

Sort:

By area:

  1. 2023SCLIP: Rethinking Self-Attention for Dense Vision-Language Inferencecited by 169
  2. 2022VICRegL: Self-Supervised Learning of Local Visual Featurescited by 118
  3. 2023Bidirectional Copy-Paste for Semi-Supervised Medical Image Segmentationcited by 115
  4. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 53
  5. 2022A Survey on Label-efficient Deep Image Segmentation: Bridging the Gap between Weak Supervision and Dense Predictioncited by 46
  6. 2022Learning to Decompose Visual Features with Latent Textual Promptscited by 28
  7. 2023CrOC: Cross-View Online Clustering for Dense Visual Representation Learningcited by 21
  8. 2023CrIBo: Self-Supervised Learning via Cross-Image Object-Level Bootstrappingcited by 17
  9. 2025How far can we go with ImageNet for Text-to-Image generation?cited by 10
  10. 2023Introspective Deep Metric Learningcited by 7
  11. 2023INoD: Injected Noise Discriminator for Self-Supervised Representation Learning in Agricultural Fieldscited by 6
  12. 2023Revisiting DETR Pre-training for Object Detectioncited by 5
14 more
  1. 2022OPERA: Omni-Supervised Representation Learning with Hierarchical Supervisionscited by 3
  2. 2023A Systematic Literature Review of Computer Vision Applications in Robotized Wire Harness Assemblycited by 3
  3. 2023Disentangled Pre-training for Image Mattingcited by 2
  4. 2023Self-supervised Learning of Contextualized Local Visual Embeddingscited by 2
  5. 2024SemanticMIM: Marring Masked Image Modeling with Semantics Compression for General Visual Representationcited by 2
  6. 2024Investigating Self-Supervised Methods for Label-Efficient Learningcited by 2
  7. 2024Self-Supervised Learning for Real-World Object Detection: a Surveycited by 2
  8. 2022Copy-Pasting Coherent Depth Regions Improves Contrastive Learning for Urban-Scene Segmentationcited by 0
  9. 2023MIMIC: Masked Image Modeling with Image Correspondencescited by 0
  10. 2024Dual-Teacher Ensemble Models with Double-Copy-Paste for 3D Semi-Supervised Medical Image Segmentationcited by 0
  11. 2025Beyond the Encoder: Joint Encoder-Decoder Contrastive Pre-Training Improves Dense Predictioncited by 0
  12. 2026Tracing Copied Pixels and Regularizing Patch Affinity in Copy Detectioncited by 0
  13. 2026Mask-supervised Object-centric Representation Learning with LeJEPAcited by 0
  14. 2026Depth-Guided Contrastive Learning for 2D Representations with 3D Spatial Awarenesscited by 0