mlsanityData to 7 Oct 2026

Bootstrap your own latent: A new approach to self-supervised Learning

arXiv 2006.07733 ↗First posted 13 Jun 2020cs.LG cs.CV stat.MLcited by 4,712

Not on the map. Why?

Cited by 95 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the same time, we update the target network with a slow-moving average of the online network. While state-of-the art methods rely on negative pairs, BYOL achieves a new state of the art without them. BYOL reaches 74.3% top-1 classification accuracy on ImageNet using a linear evaluation with a ResNet-50 architecture and 79.6% with a larger ResNet. We show that BYOL performs on par or better than the current state of the art on both transfer and semi-supervised benchmarks. Our implementation and pretrained models are given on GitHub.

Key facts. Cited by 4,712 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 684; 2022: 1068; 2023: 1068; 2024: 817; 2025: 593; 2026: 482.

4,712 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  6. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  7. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  8. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  9. 2023Consistency Modelscited by 1,778
  10. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  11. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  12. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
38 more
  1. 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
  2. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  3. 2021Transformers in Vision: A Surveycited by 1,009
  4. 2021Scaling Vision Transformerscited by 993
  5. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  6. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  7. 2021Model-Contrastive Federated Learningcited by 762
  8. 2021Domain Generalization: A Surveycited by 760
  9. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  10. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  11. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  12. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  13. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  14. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  15. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  16. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  17. 2021Extract Free Dense Labels from CLIPcited by 580
  18. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  19. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  20. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 553
  21. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  22. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  23. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 456
  24. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 447
  25. 2022GraphMAE: Self-Supervised Masked Graph Autoencoderscited by 443
  26. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 441
  27. 2023Emergent Correspondence from Image Diffusioncited by 437
  28. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 434
  29. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  30. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 429
  31. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  32. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  33. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  34. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  35. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 402
  36. 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 397
  37. 2022BEATs: Audio Pre-Training with Acoustic Tokenizerscited by 378
  38. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374

50 most cited of 4,712 All 4,712 (JSON)