# An Effective Weight Initialization Method for Deep Learning: Application to Satellite Image Classification

arXiv: https://arxiv.org/abs/2406.00348 · HTML: https://mlsanity.com/paper/2406.00348/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 38 references that are in the map.
- First posted on arXiv 1 Jun 2024.

## Builds on (most cited first, 13 of 13)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Deeply-Supervised Nets](https://mlsanity.com/paper/1409.5185/) (2014)
- [AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification](https://mlsanity.com/paper/1608.05167/) (2016)
- [All you need is a good init](https://mlsanity.com/paper/1511.06422/) (2015)
- [PatternNet: A Benchmark Dataset for Performance Evaluation of Remote Sensing Image Retrieval](https://mlsanity.com/paper/1706.03424/) (2017)
- [ZerO Initialization: Initializing Neural Networks with only Zeros and Ones](https://mlsanity.com/paper/2110.12661/) (2021)
- [Remote Sensing Change Detection With Transformers Trained from Scratch](https://mlsanity.com/paper/2304.06710/) (2023)
- [Random Walk Initialization for Training Very Deep Feedforward Networks](https://mlsanity.com/paper/1412.6558/) (2014)
- [Randomly Initialized Convolutional Neural Network for the Recognition of COVID-19 using X-ray Images](https://mlsanity.com/paper/2105.08199/) (2021)
- [Weight Initialization Techniques for Deep Learning Algorithms in Remote Sensing: Recent Trends and Future Perspectives](https://mlsanity.com/paper/2102.07004/) (2021)
- [A Unified Weight Initialization Paradigm for Tensorial Convolutional Neural Networks](https://mlsanity.com/paper/2205.15307/) (2022)
- [Low-Shot Learning for the Semantic Segmentation of Remote Sensing Imagery](https://mlsanity.com/paper/1803.09824/) (2018)

## Built on by (most cited first, 1 of 1)

- [MedNNS: Supernet-based Medical Task-Adaptive Neural Network Search](https://mlsanity.com/paper/2504.15865/) (2025)

## Abstract (arXiv)

The growing interest in satellite imagery has triggered the need for efficient mechanisms to extract valuable information from these vast data sources, providing deeper insights. Even though deep learning has shown significant progress in satellite image classification. Nevertheless, in the literature, only a few results can be found on weight initialization techniques. These techniques traditionally involve initializing the networks' weights before training on extensive datasets, distinct from fine-tuning the weights of pre-trained networks. In this study, a novel weight initialization method is proposed in the context of satellite image classification. The proposed weight initialization method is mathematically detailed during the forward and backward passes of the convolutional neural network (CNN) model. Extensive experiments are carried out using six real-world datasets. Comparative analyses with existing weight initialization techniques made on various well-known CNN models reveal that the proposed weight initialization technique outperforms the previous competitive techniques in classification accuracy. The complete code of the proposed technique, along with the obtained results, is available at https://github.com/WadiiBoulila/Weight-Initialization