# TriBYOL: Triplet BYOL for Self-Supervised Representation Learning

arXiv: https://arxiv.org/abs/2206.03012 · HTML: https://mlsanity.com/paper/2206.03012/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 7 Jun 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Deep Learning for Classical Japanese Literature](https://mlsanity.com/paper/1812.01718/) (2018)
- [Understanding self-supervised Learning Dynamics without Contrastive Pairs](https://mlsanity.com/paper/2102.06810/) (2021)
- [Understanding Self-supervised Learning with Dual Deep Networks](https://mlsanity.com/paper/2010.00578/) (2020)
- [Remote Sensing Image Scene Classification with Self-Supervised Paradigm under Limited Labeled Samples](https://mlsanity.com/paper/2010.00882/) (2020)

## Built on by (most cited first, 8 of 8)

- [Compressed Gastric Image Generation Based on Soft-Label Dataset Distillation for Medical Data Sharing](https://mlsanity.com/paper/2209.14635/) (2022)
- [Dataset Distillation Using Parameter Pruning](https://mlsanity.com/paper/2209.14609/) (2022)
- [Importance-Aware Adaptive Dataset Distillation](https://mlsanity.com/paper/2401.15863/) (2024)
- [COVID-19 Detection Based on Self-Supervised Transfer Learning Using Chest X-Ray Images](https://mlsanity.com/paper/2212.09276/) (2022)
- [Heterogeneous Tri-stream Clustering Network](https://mlsanity.com/paper/2301.04451/) (2023)
- [RGMIM: Region-Guided Masked Image Modeling for Learning Meaningful Representations from X-Ray Images](https://mlsanity.com/paper/2211.00313/) (2022)
- [Privacy-Aware Continual Self-Supervised Learning on Multi-Window Chest Computed Tomography for Domain-Shift Robustness](https://mlsanity.com/paper/2510.27213/) (2025)
- [BYOLMed3D: Self-Supervised Representation Learning of Medical Videos using Gradient Accumulation Assisted 3D BYOL Framework](https://mlsanity.com/paper/2208.00444/) (2022)

## Abstract (arXiv)

This paper proposes a novel self-supervised learning method for learning better representations with small batch sizes. Many self-supervised learning methods based on certain forms of the siamese network have emerged and received significant attention. However, these methods need to use large batch sizes to learn good representations and require heavy computational resources. We present a new triplet network combined with a triple-view loss to improve the performance of self-supervised representation learning with small batch sizes. Experimental results show that our method can drastically outperform state-of-the-art self-supervised learning methods on several datasets in small-batch cases. Our method provides a feasible solution for self-supervised learning with real-world high-resolution images that uses small batch sizes.