mlsanityData to 7 Oct 2026

Self-Supervised Visual Representations Learning by Contrastive Mask Prediction

arXiv 2108.07954 ↗First posted 18 Aug 2021cs.CVcited by 31

Not on the map. Why?

Abstract (arXiv)

Advanced self-supervised visual representation learning methods rely on the instance discrimination (ID) pretext task. We point out that the ID task has an implicit semantic consistency (SC) assumption, which may not hold in unconstrained datasets. In this paper, we propose a novel contrastive mask prediction (CMP) task for visual representation learning and design a mask contrast (MaskCo) framework to implement the idea. MaskCo contrasts region-level features instead of view-level features, which makes it possible to identify the positive sample without any assumptions. To solve the domain gap between masked and unmasked features, we design a dedicated mask prediction head in MaskCo. This module is shown to be the key to the success of the CMP. We evaluated MaskCo on training datasets beyond ImageNet and compare its performance with MoCo V2. Results show that MaskCo achieves comparable performance with MoCo V2 using ImageNet training dataset, but demonstrates a stronger performance across a range of downstream tasks when COCO or Conceptual Captions are used for training. MaskCo provides a promising alternative to the ID-based methods for self-supervised learning in the wild.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 18 Aug 2021.

Builds on

28 of 34 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  8. 2017Mask R-CNNcited by 7,259
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  11. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  12. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
16 more
  1. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  2. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  3. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  4. 2016Colorful Image Colorizationcited by 1,113
  5. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  6. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  7. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  8. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  9. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  10. 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
  11. 2019Scaling and Benchmarking Self-Supervised Visual Representation Learningcited by 201
  12. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  13. 2020Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biasescited by 143
  14. 2020Object Relational Graph with Teacher-Recommended Learning for Video Captioningcited by 106
  15. 2020Self-Supervised Visual Representation Learning from Hierarchical Groupingcited by 63
  16. 2019Context-Aware Visual Policy Network for Fine-Grained Image Captioningcited by 29

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 9; 2023: 9; 2024: 6; 2025: 4.

31 papers.

Sort:

  1. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  2. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  3. 2022CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completioncited by 158
  4. 2021Unsupervised Part Discovery from Contrastive Reconstructioncited by 57
  5. 2022ProposalContrast: Unsupervised Pre-training for LiDAR-based 3D Object Detectioncited by 43
  6. 2022Rethinking the Augmentation Module in Contrastive Learning: Learning Hierarchical Augmentation Invariance with Expanded Viewscited by 37
  7. 2022UniVIP: A Unified Framework for Self-Supervised Visual Pre-trainingcited by 30
  8. 2024Masked Image Modeling: A Surveycited by 28
  9. 2023Multi-Label Self-Supervised Learning with Scene Imagescited by 17
  10. 2023AKVSR: Audio Knowledge Empowered Visual Speech Recognition by Compressing Audio Knowledge of a Pretrained Modelcited by 16
  11. 2024One Leaf Reveals the Season: Occlusion-Based Contrastive Learning with Semantic-Aware Views for Efficient Visual Representationcited by 16
  12. 2022Panoramic Panoptic Segmentation: Insights Into Surrounding Parsing for Mobile Agents via Unsupervised Contrastive Learningcited by 14
19 more
  1. 2024Brain Mapping with Dense Features: Grounding Cortical Semantic Selectivity in Natural Images With Vision Transformerscited by 13
  2. 2022SERE: Exploring Feature Self-relation for Self-supervised Transformercited by 12
  3. 2021RegionCL: Can Simple Region Swapping Contribute to Contrastive Learning?cited by 11
  4. 2023Self-Supervised Representation Learning from Temporal Ordering of Automated Driving Sequencescited by 9
  5. 2023Siamese DETRcited by 7
  6. 2023Coarse Is Better? A New Pipeline Towards Self-Supervised Learning with Uncurated Imagescited by 5
  7. 2025Conditional Representation Learning for Customized Taskscited by 5
  8. 2022Beyond Instance Discrimination: Relation-aware Contrastive Self-supervised Learningcited by 4
  9. 2022Self-Supervised Pyramid Representation Learning for Multi-Label Visual Analysis and Beyondcited by 3
  10. 2023Enhancing Representations through Heterogeneous Self-Supervised Learningcited by 2
  11. 2024Self-Supervised Learning for Real-World Object Detection: a Surveycited by 2
  12. 2022Contrastive Embedding Distribution Refinement and Entropy-Aware Attention for 3D Point Cloud Classificationcited by 1
  13. 2024TraIL-Det: Transformation-Invariant Local Feature Networks for 3D LiDAR Object Detection with Unsupervised Pre-Trainingcited by 1
  14. 2025Beyond Instance Consistency: Investigating View Diversity in Self-supervised Learningcited by 1
  15. 2025Reducing Label Dependency in Human Activity Recognition with Wearables: From Supervised Learning to Novel Weakly Self-Supervised Approachescited by 1
  16. 2023Evaluating The Robustness of Self-Supervised Representations to Background/Foreground Removalcited by 0
  17. 2023Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Workcited by 0
  18. 2024PR-MIM: Delving Deeper into Partial Reconstruction in Masked Image Modelingcited by 0
  19. 2025Self-Supervised Contrastive Learning for Multi-Label Imagescited by 0