# Initialization Using Perlin Noise for Training Networks with a Limited Amount of Data

arXiv: https://arxiv.org/abs/2101.07406 · HTML: https://mlsanity.com/paper/2101.07406/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 19 Jan 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Learning Combinations of Activation Functions](https://mlsanity.com/paper/1801.09403/) (2018)
- [Selective Unsupervised Feature Learning with Convolutional Neural Network (S-CNN)](https://mlsanity.com/paper/1606.02210/) (2016)

## Built on by (most cited first, 7 of 7)

- [Pre-training Vision Transformers with Formula-driven Supervised Learning](https://mlsanity.com/paper/2206.09132/) (2022)
- [Visual Atoms: Pre-training Vision Transformers with Sinusoidal Waves](https://mlsanity.com/paper/2303.01112/) (2023)
- [Scaling Backwards: Minimal Synthetic Pre-training?](https://mlsanity.com/paper/2408.00677/) (2024)
- [Primitive Geometry Segment Pre-training for 3D Medical Image Segmentation](https://mlsanity.com/paper/2401.03665/) (2024)
- [Deep Learning of Crystalline Defects from TEM images: A Solution for the Problem of "Never Enough Training Data"](https://mlsanity.com/paper/2307.06322/) (2023)
- [Formula-Supervised Visual-Geometric Pre-training](https://mlsanity.com/paper/2409.13535/) (2024)
- [MoireDB: Formula-generated Interference-fringe Image Dataset](https://mlsanity.com/paper/2502.01490/) (2025)

## Abstract (arXiv)

We propose a novel network initialization method using Perlin noise for training image classification networks with a limited amount of data. Our main idea is to initialize the network parameters by solving an artificial noise classification problem, where the aim is to classify Perlin noise samples into their noise categories. Specifically, the proposed method consists of two steps. First, it generates Perlin noise samples with category labels defined based on noise complexity. Second, it solves a classification problem, in which network parameters are optimized to classify the generated noise samples. This method produces a reasonable set of initial weights (filters) for image classification. To the best of our knowledge, this is the first work to initialize networks by solving an artificial optimization problem without using any real-world images. Our experiments show that the proposed method outperforms conventional initialization methods on four image classification datasets.