# Self-Supervised learning for Neural Architecture Search (NAS)

arXiv: https://arxiv.org/abs/2304.01023 · HTML: https://mlsanity.com/paper/2304.01023/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018)
- [Multi-Task Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2009.09796/) (2020)
- [How Does Batch Normalization Help Optimization?](https://mlsanity.com/paper/1805.11604/) (2018)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [Deep Learning on Image Denoising: An overview](https://mlsanity.com/paper/1912.13171/) (2019)
- [An Overview of Deep Semi-Supervised Learning](https://mlsanity.com/paper/2006.05278/) (2020)
- [ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution](https://mlsanity.com/paper/1907.09837/) (2019)
- [A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising](https://mlsanity.com/paper/1807.04364/) (2018)
- [Iterative Bounding Box Annotation for Object Detection](https://mlsanity.com/paper/2007.00961/) (2020)
- [Deep Structured Prediction for Facial Landmark Detection](https://mlsanity.com/paper/2010.09035/) (2020)
- [Do Normalization Layers in a Deep ConvNet Really Need to Be Distinct?](https://mlsanity.com/paper/1811.07727/) (2018)

## Abstract (arXiv)

The objective of this internship is to propose an innovative method that uses unlabelled data, i.e. data that will allow the AI to automatically learn to predict the correct outcome. To reach this stage, the steps to be followed can be defined as follows: (1) consult the state of the art and position ourself against it, (2) come up with ideas for development paths, (3) implement these ideas, (4) and finally test them to position ourself against the state of the art, and then start the sequence again. During my internship, this sequence was done several times and therefore gives the tracks explored during the internship.