# Run Away From your Teacher: Understanding BYOL by a Novel Self-Supervised Approach

arXiv: https://arxiv.org/abs/2011.10944 · HTML: https://mlsanity.com/paper/2011.10944/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2020.

## Built on by (most cited first, 12 of 12)

- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods](https://mlsanity.com/paper/2205.11508/) (2022)
- [Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning](https://mlsanity.com/paper/2105.05682/) (2021)
- [SGL-PT: A Strong Graph Learner with Graph Prompt Tuning](https://mlsanity.com/paper/2302.12449/) (2023)
- [A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm](https://mlsanity.com/paper/2310.12244/) (2023)
- [MARIO: Model Agnostic Recipe for Improving OOD Generalization of Graph Contrastive Learning](https://mlsanity.com/paper/2307.13055/) (2023)
- [Hyperspherically Regularized Networks for Self-Supervision](https://mlsanity.com/paper/2105.00925/) (2021)
- [Cross-Model Cross-Stream Learning for Self-Supervised Human Action Recognition](https://mlsanity.com/paper/2307.07791/) (2023)
- [Self-Labeling Refinement for Robust Representation Learning with Bootstrap Your Own Latent](https://mlsanity.com/paper/2204.04545/) (2022)
- [Structure-Aware Group Discrimination with Adaptive-View Graph Encoder: A Fast Graph Contrastive Learning Framework](https://mlsanity.com/paper/2303.05231/) (2023)
- [BioNCERE: Non-Contrastive Enhancement For Relation Extraction In Biomedical Texts](https://mlsanity.com/paper/2410.23583/) (2024)
- [Self-Supervised Representation Learning as Mutual Information Maximization](https://mlsanity.com/paper/2510.01345/) (2025)

## Abstract (arXiv)

Recently, a newly proposed self-supervised framework Bootstrap Your Own Latent (BYOL) seriously challenges the necessity of negative samples in contrastive learning frameworks. BYOL works like a charm despite the fact that it discards the negative samples completely and there is no measure to prevent collapse in its training objective. In this paper, we suggest understanding BYOL from the view of our proposed interpretable self-supervised learning framework, Run Away From your Teacher (RAFT). RAFT optimizes two objectives at the same time: (i) aligning two views of the same data to similar representations and (ii) running away from the model's Mean Teacher (MT, the exponential moving average of the history models) instead of BYOL's running towards it. The second term of RAFT explicitly prevents the representation collapse and thus makes RAFT a more conceptually reliable framework. We provide basic benchmarks of RAFT on CIFAR10 to validate the effectiveness of our method. Furthermore, we prove that BYOL is equivalent to RAFT under certain conditions, providing solid reasoning for BYOL's counter-intuitive success.