mlsanityData to 7 Oct 2026

Unsupervised Learning by Predicting Noise

arXiv 1704.05310 ↗First posted 18 Apr 2017stat.ML cs.CV cs.LGcited by 97

Not on the map. Why?

Abstract (arXiv)

Convolutional neural networks provide visual features that perform remarkably well in many computer vision applications. However, training these networks requires significant amounts of supervision. This paper introduces a generic framework to train deep networks, end-to-end, with no supervision. We propose to fix a set of target representations, called Noise As Targets (NAT), and to constrain the deep features to align to them. This domain agnostic approach avoids the standard unsupervised learning issues of trivial solutions and collapsing of features. Thanks to a stochastic batch reassignment strategy and a separable square loss function, it scales to millions of images. The proposed approach produces representations that perform on par with state-of-the-art unsupervised methods on ImageNet and Pascal VOC.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 34; 2022: 25; 2023: 21; 2024: 14; 2025: 0; 2026: 3.

97 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  4. 2023A Cookbook of Self-Supervised Learningcited by 248
  5. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  6. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  7. 2021Self-supervised Pretraining of Visual Features in the Wildcited by 197
  8. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  9. 2021SCARF: Self-Supervised Contrastive Learning using Random Feature Corruptioncited by 151
  10. 2024Scalable Pre-training of Large Autoregressive Image Modelscited by 115
  11. 2022Vertical Federated Learning: Concepts, Advances and Challengescited by 104
  12. 2021SiT: Self-supervised vIsion Transformercited by 99
38 more
  1. 2021Large-scale Unsupervised Semantic Segmentationcited by 98
  2. 2021Image Quality Assessment using Contrastive Learningcited by 95
  3. 2023CALM: Conditional Adversarial Latent Models for Directable Virtual Characterscited by 91
  4. 2022Self-Supervised Learning of Object Parts for Semantic Segmentationcited by 90
  5. 2021Divide and Contrast: Self-supervised Learning from Uncurated Datacited by 85
  6. 2022Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervisioncited by 80
  7. 2022Masked Frequency Modeling for Self-Supervised Visual Pre-Trainingcited by 70
  8. 2021Clustering-friendly Representation Learning via Instance Discrimination and Feature Decorrelationcited by 65
  9. 2021What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?cited by 63
  10. 2022Discovering Object Masks with Transformers for Unsupervised Semantic Segmentationcited by 54
  11. 2021Pareto Self-Supervised Training for Few-Shot Learningcited by 52
  12. 2024Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approachcited by 48
  13. 2022Non-isotropy Regularization for Proxy-based Deep Metric Learningcited by 33
  14. 2022Self-supervised Learning from 100 Million Medical Imagescited by 31
  15. 2021Multimodal Representation Learning via Maximization of Local Mutual Informationcited by 27
  16. 2021Simpler, Faster, Stronger: Breaking The log-K Curse On Contrastive Learners With FlatNCEcited by 22
  17. 2022Practical Vertical Federated Learning with Unsupervised Representation Learningcited by 19
  18. 2021Cluster-guided Asymmetric Contrastive Learning for Unsupervised Person Re-Identificationcited by 18
  19. 2023Learning to Estimate 6DoF Pose from Limited Data: A Few-Shot, Generalizable Approach using RGB Imagescited by 18
  20. 2021Self-Adaptive Training: Bridging Supervised and Self-Supervised Learningcited by 17
  21. 2021Nearest Neighborhood-Based Deep Clustering for Source Data-absent Unsupervised Domain Adaptationcited by 16
  22. 2021Modelling Neighbor Relation in Joint Space-Time Graph for Video Correspondence Learningcited by 16
  23. 2024You Don't Need Domain-Specific Data Augmentations When Scaling Self-Supervised Learningcited by 16
  24. 2022MetAug: Contrastive Learning via Meta Feature Augmentationcited by 15
  25. 2021SimSR: Simple Distance-based State Representation for Deep Reinforcement Learningcited by 14
  26. 2022MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Seriescited by 12
  27. 2021Federated Learning of User Verification Models Without Sharing Embeddingscited by 11
  28. 2021Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Frameworkcited by 11
  29. 2023Correlational Image Modeling for Self-Supervised Visual Pre-Trainingcited by 11
  30. 2023Test Time Adaptation for Blind Image Quality Assessmentcited by 11
  31. 2021MC-SSL0.0: Towards Multi-Concept Self-Supervised Learningcited by 10
  32. 2024Trajectory-aligned Space-time Tokens for Few-shot Action Recognitioncited by 10
  33. 2021ScatSimCLR: self-supervised contrastive learning with pretext task regularization for small-scale datasetscited by 9
  34. 2022Unsupervised pre-training of graph transformers on patient population graphscited by 9
  35. 2022Global-Local Self-Distillation for Visual Representation Learningcited by 8
  36. 2024The Common Stability Mechanism behind most Self-Supervised Learning Approachescited by 8
  37. 2024Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representationscited by 8
  38. 2024Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?cited by 8

50 most cited of 97 All 97 (JSON)