# Concept Representation Learning with Contrastive Self-Supervised Learning

arXiv: https://arxiv.org/abs/2112.05677 · HTML: https://mlsanity.com/paper/2112.05677/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Builds on (most cited first, 12 of 12)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Class-incremental learning: survey and performance evaluation on image classification](https://mlsanity.com/paper/2010.15277/) (2020)
- [Essentials for Class Incremental Learning](https://mlsanity.com/paper/2102.09517/) (2021)
- [Self-Supervised Relational Reasoning for Representation Learning](https://mlsanity.com/paper/2006.05849/) (2020)
- [Rethinking Self-Supervised Learning: Small is Beautiful](https://mlsanity.com/paper/2103.13559/) (2021)
- [Self-Supervised Class Incremental Learning](https://mlsanity.com/paper/2111.11208/) (2021)
- [Concept-Oriented Deep Learning](https://mlsanity.com/paper/1806.01756/) (2018)
- [Tiered Graph Autoencoders with PyTorch Geometric for Molecular Graphs](https://mlsanity.com/paper/1908.08612/) (2019)

## Built on by (most cited first, 4 of 4)

- [Exemplar-Based Contrastive Self-Supervised Learning with Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2202.02601/) (2022)
- [Dual Embodied-Symbolic Concept Representations for Deep Learning](https://mlsanity.com/paper/2203.00600/) (2022)
- [Advancing Perception in Artificial Intelligence through Principles of Cognitive Science](https://mlsanity.com/paper/2310.08803/) (2023)
- [Embodied-Symbolic Contrastive Graph Self-Supervised Learning for Molecular Graphs](https://mlsanity.com/paper/2205.06783/) (2022)

## Abstract (arXiv)

Concept-oriented deep learning (CODL) is a general approach to meet the future challenges for deep learning: (1) learning with little or no external supervision, (2) coping with test examples that come from a different distribution than the training examples, and (3) integrating deep learning with symbolic AI. In CODL, as in human learning, concept representations are learned based on concept exemplars. Contrastive self-supervised learning (CSSL) provides a promising approach to do so, since it: (1) uses data-driven associations, to get away from semantic labels, (2) supports incremental and continual learning, to get away from (large) fixed datasets, and (3) accommodates emergent objectives, to get away from fixed objectives (tasks). We discuss major aspects of concept representation learning using CSSL. These include dual-level concept representations, CSSL for feature representations, exemplar similarity measures and self-supervised relational reasoning, incremental and continual CSSL, and contrastive self-supervised concept (class) incremental learning. The discussion leverages recent findings from cognitive neural science and CSSL.