mlsanityData to 7 Oct 2026

CASTing Your Model: Learning to Localize Improves Self-Supervised Representations

arXiv 2012.04630 ↗First posted 8 Dec 2020cs.CV cs.AI cs.LGcited by 63

Not on the map. Why?

Abstract (arXiv)

Recent advances in self-supervised learning (SSL) have largely closed the gap with supervised ImageNet pretraining. Despite their success these methods have been primarily applied to unlabeled ImageNet images, and show marginal gains when trained on larger sets of uncurated images. We hypothesize that current SSL methods perform best on iconic images, and struggle on complex scene images with many objects. Analyzing contrastive SSL methods shows that they have poor visual grounding and receive poor supervisory signal when trained on scene images. We propose Contrastive Attention-Supervised Tuning(CAST) to overcome these limitations. CAST uses unsupervised saliency maps to intelligently sample crops, and to provide grounding supervision via a Grad-CAM attention loss. Experiments on COCO show that CAST significantly improves the features learned by SSL methods on scene images, and further experiments show that CAST-trained models are more robust to changes in backgrounds.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 21; 2023: 13; 2024: 9; 2025: 4; 2026: 3.

63 papers.

Sort:

By area:

  1. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  2. 2021Unsupervised Object-Level Representation Learning from Scene Imagescited by 72
  3. 2022Self-Supervised Visual Representation Learning with Semantic Groupingcited by 70
  4. 2022Crafting Better Contrastive Views for Siamese Representation Learningcited by 67
  5. 2022Hyperbolic Contrastive Learning for Visual Representations beyond Objectscited by 58
  6. 2021Revisiting Contrastive Methods for Unsupervised Learning of Visual Representationscited by 48
  7. 2022Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformerscited by 39
  8. 2021Object-aware Contrastive Learning for Debiased Scene Representationcited by 34
  9. 2023CLIP-S4: Language-Guided Self-Supervised Semantic Segmentationcited by 34
  10. 2023Learning Visual Representations via Language-Guided Samplingcited by 32
  11. 2022Architecture-Agnostic Masked Image Modeling -- From ViT back to CNNcited by 31
  12. 2022UniVIP: A Unified Framework for Self-Supervised Visual Pre-trainingcited by 30
38 more
  1. 2024EAGLE: Eigen Aggregation Learning for Object-Centric Unsupervised Semantic Segmentationcited by 27
  2. 2023Anatomical Invariance Modeling and Semantic Alignment for Self-supervised Learning in 3D Medical Image Analysiscited by 25
  3. 2021Augmentations in Graph Contrastive Learning: Current Methodological Flaws & Towards Better Practicescited by 24
  4. 2022Improving Visual Grounding by Encouraging Consistent Gradient-based Explanationscited by 24
  5. 2021Object-Aware Cropping for Self-Supervised Learningcited by 22
  6. 2023CrOC: Cross-View Online Clustering for Dense Visual Representation Learningcited by 21
  7. 2023Boosting Video Object Segmentation via Space-time Correspondence Learningcited by 21
  8. 2021Look at What I'm Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videoscited by 18
  9. 2022Learning Hierarchical Image Segmentation For Recognition and By Recognitioncited by 16
  10. 2023Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problemscited by 14
  11. 2022Semantic-Aware Fine-Grained Correspondencecited by 13
  12. 2022Exploring the Coordination of Frequency and Attention in Masked Image Modelingcited by 12
  13. 2023Correlational Image Modeling for Self-Supervised Visual Pre-Trainingcited by 11
  14. 2024PooDLe: Pooled and dense self-supervised learning from naturalistic videoscited by 11
  15. 2024Understanding Hyperbolic Metric Learning through Hard Negative Samplingcited by 10
  16. 2021Exploring Localization for Self-supervised Fine-grained Contrastive Learningcited by 9
  17. 2021Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentationscited by 8
  18. 2022Sim2Real Object-Centric Keypoint Detection and Descriptioncited by 8
  19. 2022Dense Siamese Network for Dense Unsupervised Learningcited by 8
  20. 2022Can domain adaptation make object recognition work for everyone?cited by 8
  21. 2024GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learningcited by 8
  22. 2023Understanding Self-Supervised Features for Learning Unsupervised Instance Segmentationcited by 7
  23. 2023Mining Gaze for Contrastive Learning toward Computer-Assisted Diagnosiscited by 7
  24. 2021GenURL: A General Framework for Unsupervised Representation Learningcited by 6
  25. 2022Multi-Task Learning for Visual Scene Understandingcited by 6
  26. 2022Towards Efficient and Effective Self-Supervised Learning of Visual Representationscited by 6
  27. 2022Refine and Represent: Region-to-Object Representation Learningcited by 5
  28. 2021Semi-weakly Supervised Contrastive Representation Learning for Retinal Fundus Imagescited by 4
  29. 2023Learning Better Contrastive View from Radiologist's Gazecited by 4
  30. 2023Mitigating Bias: Enhancing Image Classification by Improving Model Explanationscited by 3
  31. 2021ExCon: Explanation-driven Supervised Contrastive Learning for Image Classificationcited by 2
  32. 2021PreViTS: Contrastive Pretraining with Video Tracking Supervisioncited by 2
  33. 2022MSR: Making Self-supervised learning Robust to Aggressive Augmentationscited by 2
  34. 2023Saliency Guided Contrastive Learning on Scene Imagescited by 2
  35. 2024On Learning Discriminative Features from Synthesized Data for Self-Supervised Fine-Grained Visual Recognitioncited by 2
  36. 2024Self-Supervised Learning for Real-World Object Detection: a Surveycited by 2
  37. 2022CYBORGS: Contrastively Bootstrapping Object Representations by Grounding in Segmentationcited by 1
  38. 2022On the robustness of self-supervised representations for multi-view object classificationcited by 1

50 most cited of 63 All 63 (JSON)