# Towards evolution of Deep Neural Networks through contrastive Self-Supervised learning

arXiv: https://arxiv.org/abs/2406.14525 · HTML: https://mlsanity.com/paper/2406.14525/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 20 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Large Batch Training of Convolutional Networks](https://mlsanity.com/paper/1708.03888/) (2017)
- [Evolving Deep Neural Networks](https://mlsanity.com/paper/1703.00548/) (2017)
- [Evolving Losses for Unsupervised Video Representation Learning](https://mlsanity.com/paper/2002.12177/) (2020)
- [Self-supervised learning through the eyes of a child](https://mlsanity.com/paper/2007.16189/) (2020)
- [Evolutionary Generative Adversarial Networks](https://mlsanity.com/paper/1803.00657/) (2018)
- [DENSER: Deep Evolutionary Network Structured Representation](https://mlsanity.com/paper/1801.01563/) (2018)
- [Self-supervised Representation Learning for Evolutionary Neural Architecture Search](https://mlsanity.com/paper/2011.00186/) (2020)
- [Exploring the Evolution of GANs through Quality Diversity](https://mlsanity.com/paper/2007.06251/) (2020)
- [Genetic Algorithms for Evolving Deep Neural Networks](https://mlsanity.com/paper/1711.07655/) (2017)
- [Deciphering the Projection Head: Representation Evaluation Self-supervised Learning](https://mlsanity.com/paper/2301.12189/) (2023)
- [Coevolution of Generative Adversarial Networks](https://mlsanity.com/paper/1912.06172/) (2019)
- [Evolving GAN Formulations for Higher Quality Image Synthesis](https://mlsanity.com/paper/2102.08578/) (2021)
- [Evolutionary Augmentation Policy Optimization for Self-supervised Learning](https://mlsanity.com/paper/2303.01584/) (2023)
- [Using Skill Rating as Fitness on the Evolution of GANs](https://mlsanity.com/paper/2004.04796/) (2020)

## Abstract (arXiv)

Deep Neural Networks (DNNs) have been successfully applied to a wide range of problems. However, two main limitations are commonly pointed out. The first one is that they require long time to design. The other is that they heavily rely on labelled data, which can sometimes be costly and hard to obtain. In order to address the first problem, neuroevolution has been proved to be a plausible option to automate the design of DNNs. As for the second problem, self-supervised learning has been used to leverage unlabelled data to learn representations. Our goal is to study how neuroevolution can help self-supervised learning to bridge the gap to supervised learning in terms of performance. In this work, we propose a framework that is able to evolve deep neural networks using self-supervised learning. Our results on the CIFAR-10 dataset show that it is possible to evolve adequate neural networks while reducing the reliance on labelled data. Moreover, an analysis to the structure of the evolved networks suggests that the amount of labelled data fed to them has less effect on the structure of networks that learned via self-supervised learning, when compared to individuals that relied on supervised learning.