# Transitive Invariance for Self-supervised Visual Representation Learning

arXiv: https://arxiv.org/abs/1708.02901 · HTML: https://mlsanity.com/paper/1708.02901/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2017.

## Built on by (most cited first, 30 of 37)

- [Masked Feature Prediction for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2112.09133/) (2021)
- [A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning](https://mlsanity.com/paper/2104.14558/) (2021)
- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [Self-supervised Learning is More Robust to Dataset Imbalance](https://mlsanity.com/paper/2110.05025/) (2021)
- [Mixed supervision for surface-defect detection: from weakly to fully supervised learning](https://mlsanity.com/paper/2104.06064/) (2021)
- [Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision](https://mlsanity.com/paper/2202.08360/) (2022)
- [Diffusion Models as Masked Autoencoders](https://mlsanity.com/paper/2304.03283/) (2023)
- [Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contexts](https://mlsanity.com/paper/2109.04379/) (2021)
- [Jigsaw Clustering for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2104.00323/) (2021)
- [EgoEnv: Human-centric environment representations from egocentric video](https://mlsanity.com/paper/2207.11365/) (2022)
- [Progressive Video Summarization via Multimodal Self-supervised Learning](https://mlsanity.com/paper/2201.02494/) (2022)
- [ViewCLR: Learning Self-supervised Video Representation for Unseen Viewpoints](https://mlsanity.com/paper/2112.03905/) (2021)
- [Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos](https://mlsanity.com/paper/2308.09951/) (2023)
- [Salient Objects in Clutter](https://mlsanity.com/paper/2105.03053/) (2021)
- [Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency](https://mlsanity.com/paper/2105.06463/) (2021)
- [RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning](https://mlsanity.com/paper/2204.11167/) (2022)
- [Facial-Sketch Synthesis: A New Challenge](https://mlsanity.com/paper/2112.15439/) (2021)
- [SERE: Exploring Feature Self-relation for Self-supervised Transformer](https://mlsanity.com/paper/2206.05184/) (2022)
- [Analysis of Augmentations for Contrastive ECG Representation Learning](https://mlsanity.com/paper/2206.07656/) (2022)
- [See Yourself in Others: Attending Multiple Tasks for Own Failure Detection](https://mlsanity.com/paper/2110.02549/) (2021)
- [SOS! Self-supervised Learning Over Sets Of Handled Objects In Egocentric Action Recognition](https://mlsanity.com/paper/2204.04796/) (2022)
- [Visual Pre-training for Navigation: What Can We Learn from Noise?](https://mlsanity.com/paper/2207.00052/) (2022)
- [Deep adaptive fuzzy clustering for evolutionary unsupervised representation learning](https://mlsanity.com/paper/2103.17086/) (2021)
- [A Large-Scale Analysis on Self-Supervised Video Representation Learning](https://mlsanity.com/paper/2306.06010/) (2023)
- [Pixel-level Correspondence for Self-Supervised Learning from Video](https://mlsanity.com/paper/2207.03866/) (2022)
- [Local Manifold Augmentation for Multiview Semantic Consistency](https://mlsanity.com/paper/2211.02798/) (2022)
- [A Large-Scale Analysis on Contextual Self-Supervised Video Representation Learning](https://mlsanity.com/paper/2504.06153/) (2025)
- [RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image Generation](https://mlsanity.com/paper/2504.17502/) (2025)
- [AugNet: End-to-End Unsupervised Visual Representation Learning with Image Augmentation](https://mlsanity.com/paper/2106.06250/) (2021)
- [PreViTS: Contrastive Pretraining with Video Tracking Supervision](https://mlsanity.com/paper/2112.00804/) (2021)

## Abstract (arXiv)

Learning visual representations with self-supervised learning has become popular in computer vision. The idea is to design auxiliary tasks where labels are free to obtain. Most of these tasks end up providing data to learn specific kinds of invariance useful for recognition. In this paper, we propose to exploit different self-supervised approaches to learn representations invariant to (i) inter-instance variations (two objects in the same class should have similar features) and (ii) intra-instance variations (viewpoint, pose, deformations, illumination, etc). Instead of combining two approaches with multi-task learning, we argue to organize and reason the data with multiple variations. Specifically, we propose to generate a graph with millions of objects mined from hundreds of thousands of videos. The objects are connected by two types of edges which correspond to two types of invariance: "different instances but a similar viewpoint and category" and "different viewpoints of the same instance". By applying simple transitivity on the graph with these edges, we can obtain pairs of images exhibiting richer visual invariance. We use this data to train a Triplet-Siamese network with VGG16 as the base architecture and apply the learned representations to different recognition tasks. For object detection, we achieve 63.2% mAP on PASCAL VOC 2007 using Fast R-CNN (compare to 67.3% with ImageNet pre-training). For the challenging COCO dataset, our method is surprisingly close (23.5%) to the ImageNet-supervised counterpart (24.4%) using the Faster R-CNN framework. We also show that our network can perform significantly better than the ImageNet network in the surface normal estimation task.