mlsanityData to 14 Oct 2026

Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning

arXiv 2112.04607 ↗First posted 8 Dec 2021cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

We are interested in representation learning in self-supervised, supervised, and semi-supervised settings. Some recent self-supervised learning methods like mean-shift (MSF) cluster images by pulling the embedding of a query image to be closer to its nearest neighbors (NNs). Since most NNs are close to the query by design, the averaging may not affect the embedding of the query much. On the other hand, far away NNs may not be semantically related to the query. We generalize the mean-shift idea by constraining the search space of NNs using another source of knowledge so that NNs are far from the query while still being semantically related. We show that our method (1) outperforms MSF in SSL setting when the constraint utilizes a different augmentation of an image from the previous epoch, and (2) outperforms PAWS in semi-supervised setting with less training resources when the constraint ensures that the NNs have the same pseudo-label as the query.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 65 of its 101 references that are in the map. First posted on arXiv 8 Dec 2021.

Builds on

65 of 101 references.

References · 65

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  8. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  9. 2017Squeeze-and-Excitation Networkscited by 4,789
  10. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  11. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  12. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
38 more
  1. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  2. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  3. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  4. 2020Supervised Contrastive Learningcited by 2,829
  5. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  6. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  7. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  8. 2016Matching Networks for One Shot Learningcited by 2,255
  9. 2013Describing Textures in the Wildcited by 2,125
  10. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  11. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  12. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  13. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  14. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  15. 2019Contrastive Multiview Codingcited by 1,301
  16. 2016Colorful Image Colorizationcited by 1,114
  17. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,060
  18. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  19. 2019When Does Label Smoothing Help?cited by 988
  20. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  21. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  22. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  23. 2018Born Again Neural Networkscited by 546
  24. 2019Self-labelling via simultaneous clustering and representation learningcited by 528
  25. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  26. 2020Hard Negative Mixing for Contrastive Learningcited by 402
  27. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  28. 2020Meta Pseudo Labelscited by 374
  29. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  30. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 326
  31. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 300
  32. 2017Sampling Matters in Deep Embedding Learningcited by 285
  33. 2020Self-supervised Co-training for Video Representation Learningcited by 232
  34. 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
  35. 2017Representation Learning by Learning to Countcited by 120
  36. 2020Boosting Contrastive Self-Supervised Learning with False Negative Cancellationcited by 119
  37. 2021Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samplescited by 114
  38. 2018Label Refinery: Improving ImageNet Classification through Label Progressioncited by 65

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 6; 2024: 0; 2025: 1.

7 papers.

  1. 2023Rethinking Weak Supervision in Helping Contrastive Learningcited by 18
  2. 2023Contextually Affinitive Neighborhood Refinery for Deep Clusteringcited by 10
  3. 2023Self-Supervised Representation Learning with Cross-Context Learning between Global and Hypercolumn Featurescited by 4
  4. 2025An Inclusive Theoretical Framework of Robust Supervised Contrastive Loss against Label Noisecited by 2
  5. 2023CoNe: Contrast Your Neighbours for Supervised Image Classificationcited by 1
  6. 2023MNN: Mixed Nearest-Neighbors for Self-Supervised Learningcited by 1
  7. 2023MSVQ: Self-Supervised Learning with Multiple Sample Views and Queuescited by 0