mlsanityData to 7 Oct 2026

Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Prediction

arXiv 2102.10106 ↗First posted 19 Feb 2021cs.LG stat.MLcited by 36

Not on the map. Why?

Abstract (arXiv)

State-of-the-art methods for self-supervised learning (SSL) build representations by maximizing the similarity between different transformed "views" of a sample. Without sufficient diversity in the transformations used to create views, however, it can be difficult to overcome nuisance variables in the data and build rich representations. This motivates the use of the dataset itself to find similar, yet distinct, samples to serve as views for one another. In this paper, we introduce Mine Your Own vieW (MYOW), a new approach for self-supervised learning that looks within the dataset to define diverse targets for prediction. The idea behind our approach is to actively mine views, finding samples that are neighbors in the representation space of the network, and then predict, from one sample's latent representation, the representation of a nearby sample. After showing the promise of MYOW on benchmarks used in computer vision, we highlight the power of this idea in a novel application in neuroscience where SSL has yet to be applied. When tested on multi-unit neural recordings, we find that MYOW outperforms other self-supervised approaches in all examples (in some cases by more than 10%), and often surpasses the supervised baseline. With MYOW, we show that it is possible to harness the diversity of the data to build rich views and leverage self-supervision in new domains where augmentations are limited or unknown.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 43 of its 61 references that are in the map. First posted on arXiv 19 Feb 2021.

Builds on

43 of 61 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  8. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  9. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  10. 2019Contrastive Multiview Codingcited by 1,297
  11. 2018Deep Graph Infomaxcited by 1,101
  12. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
31 more
  1. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  2. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  3. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  4. 2020Contrastive Learning with Hard Negative Samplescited by 619
  5. 2020Deep Graph Contrastive Representation Learningcited by 535
  6. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  7. 2020Hard Negative Mixing for Contrastive Learningcited by 399
  8. 2017The Effectiveness of Data Augmentation in Image Classification using Deep Learningcited by 368
  9. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  10. 2020Whitening for Self-Supervised Representation Learningcited by 278
  11. 2020What Should Not Be Contrastive in Contrastive Learningcited by 224
  12. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  13. 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
  14. 2018Markerless tracking of user-defined features with deep learningcited by 203
  15. 2020Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biasescited by 143
  16. 2019Data augmentation using learned transformations for one-shot medical image segmentationcited by 138
  17. 2020Uncovering the structure of clinical EEG signals with self-supervised learningcited by 109
  18. 2020Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learningcited by 101
  19. 2020BYOL works even without batch statisticscited by 95
  20. 2020Subject-Aware Contrastive Learning for Biosignalscited by 75
  21. 2020Understanding Self-supervised Learning with Dual Deep Networkscited by 62
  22. 2020Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAEcited by 48
  23. 2018Learning Actionable Representations from Visual Observationscited by 35
  24. 2019Self-supervised Learning for ECG-based Emotion Recognitioncited by 32
  25. 2020Multi-label Contrastive Predictive Codingcited by 32
  26. 2020Joint Contrastive Learning with Infinite Possibilitiescited by 31
  27. 2015Dropout as data augmentationcited by 27
  28. 2019Self-supervised representation learning from electroencephalography signalscited by 24
  29. 2020Extending and Analyzing Self-Supervised Learning Across Domainscited by 20
  30. 2017Learning by Association - A versatile semi-supervised training method for neural networkscited by 19
  31. 2015Anatomy-specific classification of medical images using deep convolutional netscited by 14

Built on by

Cited by year

Citing papers per year: 2021: 8; 2022: 10; 2023: 12; 2024: 3; 2025: 3.

36 papers.

Sort:

By area:

  1. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 324
  2. 2021On Feature Decorrelation in Self-Supervised Learningcited by 160
  3. 2021Unsupervised Object-Level Representation Learning from Scene Imagescited by 72
  4. 2023A Unified, Scalable Framework for Neural Population Decodingcited by 59
  5. 2021HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representationcited by 33
  6. 2022Mine yOur owN Anatomy: Revisiting Medical Image Segmentation with Extremely Limited Labelscited by 30
  7. 2023Sound Source Localization is All about Cross-Modal Alignmentcited by 30
  8. 2023S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captionscited by 29
  9. 2022The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspectivecited by 24
  10. 2021Cross-Trajectory Representation Learning for Zero-Shot Generalization in RLcited by 20
  11. 2024MIM-Refiner: A Contrastive Learning Boost from Intermediate Pre-Trained Representationscited by 19
  12. 2021Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activitycited by 18
24 more
  1. 2022Seeing the forest and the tree: Building representations of both individual and collective dynamics with transformerscited by 18
  2. 2023Soft Neighbors are Positive Supporters in Contrastive Visual Representation Learningcited by 18
  3. 2023CrIBo: Self-Supervised Learning via Cross-Image Object-Level Bootstrappingcited by 17
  4. 2022Hard Negative Sampling Strategies for Contrastive Representation Learningcited by 13
  5. 2024Aligning Sight and Sound: Advanced Sound Source Localization Through Audio-Visual Alignmentcited by 8
  6. 2025Generalizable, real-time neural decoding with hybrid state-space modelscited by 8
  7. 2021Understand and Improve Contrastive Learning Methods for Visual Representation: A Reviewcited by 7
  8. 2021Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learningcited by 7
  9. 2023MTNeuro: A Benchmark for Evaluating Representations of Brain Structure Across Multiple Levels of Abstractioncited by 7
  10. 2022Augmentation Component Analysis: Modeling Similarity via the Augmentation Overlapscited by 5
  11. 2022ProtoCLIP: Prototypical Contrastive Language Image Pretrainingcited by 5
  12. 2023Neural Latent Aligner: Cross-trial Alignment for Learning Representations of Complex, Naturalistic Neural Datacited by 5
  13. 2022Learning Behavior Representations Through Multi-Timescale Bootstrappingcited by 4
  14. 2023Relax, it doesn't matter how you get there: A new self-supervised approach for multi-timescale behavior analysiscited by 4
  15. 2022HistoPerm: A Permutation-Based View Generation Approach for Improving Histopathologic Feature Representation Learningcited by 2
  16. 2023Synthetic Hard Negative Samples for Contrastive Learningcited by 2
  17. 2025AutoSciDACT: Automated Scientific Discovery through Contrastive Embedding and Hypothesis Testingcited by 2
  18. 2025Know Thyself by Knowing Others: Learning Neuron Identity from Population Contextcited by 2
  19. 2022Learning Sinkhorn divergences for supervised change point detectioncited by 1
  20. 2022Capturing cross-session neural population variability through self-supervised identification of consistent neuron ensemblescited by 1
  21. 2023Patch-Wise Self-Supervised Visual Representation Learning: A Fine-Grained Approachcited by 1
  22. 2023Rethinking Samples Selection for Contrastive Learning: Mining of Potential Samplescited by 1
  23. 2023Predictive variational autoencoder for learning robust representations of time-series datacited by 1
  24. 2024Direct Coloring for Self-Supervised Enhanced Feature Decouplingcited by 0