# Self-Supervised Relational Reasoning for Representation Learning

arXiv: https://arxiv.org/abs/2006.05849 · HTML: https://mlsanity.com/paper/2006.05849/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2020.

## Built on by (most cited first, 30 of 37)

- [Large-scale Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2106.03149/) (2021)
- [SiT: Self-supervised vIsion Transformer](https://mlsanity.com/paper/2104.03602/) (2021)
- [Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting](https://mlsanity.com/paper/2103.13716/) (2021)
- [Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices](https://mlsanity.com/paper/2201.12382/) (2022)
- [A Review of Emerging Research Directions in Abstract Visual Reasoning](https://mlsanity.com/paper/2202.10284/) (2022)
- [LightPath: Lightweight and Scalable Path Representation Learning](https://mlsanity.com/paper/2307.10171/) (2023)
- [MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learning](https://mlsanity.com/paper/2404.13591/) (2024)
- [Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representations](https://mlsanity.com/paper/2112.02290/) (2021)
- [A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2206.04551/) (2022)
- [Self-learning for weakly supervised Gleason grading of local patterns](https://mlsanity.com/paper/2105.10420/) (2021)
- [ASiT: Local-Global Audio Spectrogram vIsion Transformer for Event Classification](https://mlsanity.com/paper/2211.13189/) (2022)
- [Learning Online Visual Invariances for Novel Objects via Supervised and Self-Supervised Training](https://mlsanity.com/paper/2110.01476/) (2021)
- [MC-SSL0.0: Towards Multi-Concept Self-Supervised Learning](https://mlsanity.com/paper/2111.15340/) (2021)
- [Remove Symmetries to Control Model Expressivity and Improve Optimization](https://mlsanity.com/paper/2408.15495/) (2024)
- [Tailoring Self-Supervision for Supervised Learning](https://mlsanity.com/paper/2207.10023/) (2022)
- [Contrasting quadratic assignments for set-based representation learning](https://mlsanity.com/paper/2205.15814/) (2022)
- [SPCXR: Self-supervised Pretraining using Chest X-rays Towards a Domain Specific Foundation Model](https://mlsanity.com/paper/2211.12944/) (2022)
- [Relational Contrastive Learning for Scene Text Recognition](https://mlsanity.com/paper/2308.00508/) (2023)
- [Concept Representation Learning with Contrastive Self-Supervised Learning](https://mlsanity.com/paper/2112.05677/) (2021)
- [DiCoM -- Diverse Concept Modeling towards Enhancing Generalizability in Chest X-Ray Studies](https://mlsanity.com/paper/2402.15534/) (2024)
- [Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unlabeled, unannotated pathology slides](https://mlsanity.com/paper/2205.01931/) (2022)
- [Class-Incremental Mixture of Gaussians for Deep Continual Learning](https://mlsanity.com/paper/2307.04094/) (2023)
- [Cooperative Decentralized Backdoor Attacks on Vertical Federated Learning](https://mlsanity.com/paper/2501.09320/) (2025)
- [Semantic Novelty Detection via Relational Reasoning](https://mlsanity.com/paper/2207.08699/) (2022)
- [A Novel Self-Learning Framework for Bladder Cancer Grading Using Histopathological Images](https://mlsanity.com/paper/2106.13559/) (2021)
- [Relational Proxies: Emergent Relationships as Fine-Grained Discriminators](https://mlsanity.com/paper/2210.02149/) (2022)
- [Gated Self-supervised Learning For Improving Supervised Learning](https://mlsanity.com/paper/2301.05865/) (2023)
- [From Patches to Objects: Exploiting Spatial Reasoning for Better Visual Representations](https://mlsanity.com/paper/2305.12384/) (2023)
- [Leveraging Hidden Structure in Self-Supervised Learning](https://mlsanity.com/paper/2106.16060/) (2021)
- [A self-training framework for glaucoma grading in OCT B-scans](https://mlsanity.com/paper/2111.11771/) (2021)

## Abstract (arXiv)

In self-supervised learning, a system is tasked with achieving a surrogate objective by defining alternative targets on a set of unlabeled data. The aim is to build useful representations that can be used in downstream tasks, without costly manual annotation. In this work, we propose a novel self-supervised formulation of relational reasoning that allows a learner to bootstrap a signal from information implicit in unlabeled data. Training a relation head to discriminate how entities relate to themselves (intra-reasoning) and other entities (inter-reasoning), results in rich and descriptive representations in the underlying neural network backbone, which can be used in downstream tasks such as classification and image retrieval. We evaluate the proposed method following a rigorous experimental procedure, using standard datasets, protocols, and backbones. Self-supervised relational reasoning outperforms the best competitor in all conditions by an average 14% in accuracy, and the most recent state-of-the-art model by 3%. We link the effectiveness of the method to the maximization of a Bernoulli log-likelihood, which can be considered as a proxy for maximizing the mutual information, resulting in a more efficient objective with respect to the commonly used contrastive losses.