# SpinalNet: Deep Neural Network with Gradual Input

arXiv: https://arxiv.org/abs/2007.03347 · HTML: https://mlsanity.com/paper/2007.03347/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2020.

## Built on by (most cited first, 30 of 40)

- [FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes](https://mlsanity.com/paper/2110.08059/) (2021)
- [Neural Monge Map estimation and its applications](https://mlsanity.com/paper/2106.03812/) (2021)
- [WaveMix: A Resource-efficient Neural Network for Image Analysis](https://mlsanity.com/paper/2205.14375/) (2022)
- [Evaluating State-of-the-Art Classification Models Against Bayes Optimality](https://mlsanity.com/paper/2106.03357/) (2021)
- [An Evolutionary Approach to Dynamic Introduction of Tasks in Large-scale Multitask Learning Systems](https://mlsanity.com/paper/2205.12755/) (2022)
- [MultiAuto-DeepONet: A Multi-resolution Autoencoder DeepONet for Nonlinear Dimension Reduction, Uncertainty Quantification and Operator Learning of Forward and Inverse Stochastic Problems](https://mlsanity.com/paper/2204.03193/) (2022)
- [WaveMix: Resource-efficient Token Mixing for Images](https://mlsanity.com/paper/2203.03689/) (2022)
- [UPANets: Learning from the Universal Pixel Attention Networks](https://mlsanity.com/paper/2103.08640/) (2021)
- [Learning Based Joint Coding-Modulation for Digital Semantic Communication Systems](https://mlsanity.com/paper/2208.05704/) (2022)
- [Generating Synthetic Datasets by Interpolating along Generalized Geodesics](https://mlsanity.com/paper/2306.06866/) (2023)
- [A Comprehensive Study on Torchvision Pre-trained Models for Fine-grained Inter-species Classification](https://mlsanity.com/paper/2110.07097/) (2021)
- [Laplace-HDC: Understanding the geometry of binary hyperdimensional computing](https://mlsanity.com/paper/2404.10759/) (2024)
- [Cross Knowledge Distillation between Artificial and Spiking Neural Networks](https://mlsanity.com/paper/2507.09269/) (2025)
- [Ranking Loss and Sequestering Learning for Reducing Image Search Bias in Histopathology](https://mlsanity.com/paper/2304.08498/) (2023)
- [Reduction of Class Activation Uncertainty with Background Information](https://mlsanity.com/paper/2305.03238/) (2023)
- [Digital Modeling on Large Kernel Metamaterial Neural Network](https://mlsanity.com/paper/2307.11862/) (2023)
- [Logical Activation Functions: Logit-space equivalents of Probabilistic Boolean Operators](https://mlsanity.com/paper/2110.11940/) (2021)
- [Conceptual Expansion Neural Architecture Search (CENAS)](https://mlsanity.com/paper/2110.03144/) (2021)
- [Signature-Graph Networks](https://mlsanity.com/paper/2110.11551/) (2021)
- [Problem-dependent attention and effort in neural networks with applications to image resolution and model selection](https://mlsanity.com/paper/2201.01415/) (2022)
- [Morphological Classification of Galaxies Using SpinalNet](https://mlsanity.com/paper/2305.01873/) (2023)
- [Deep Predictive Coding with Bi-directional Propagation for Classification and Reconstruction](https://mlsanity.com/paper/2305.18472/) (2023)
- [WSAM: Visual Explanations from Style Augmentation as Adversarial Attacker and Their Influence in Image Classification](https://mlsanity.com/paper/2308.14995/) (2023)
- [Parametric Matrix Models](https://mlsanity.com/paper/2401.11694/) (2024)
- [ScoreFusion: Fusing Score-based Generative Models via Kullback-Leibler Barycenters](https://mlsanity.com/paper/2406.19619/) (2024)
- [Intelligent Character Recognition of Handwritten Forms with Deep Neural Networks](https://mlsanity.com/paper/2606.08858/) (2026)
- [Classification of Shoulder X-Ray Images with Deep Learning Ensemble Models](https://mlsanity.com/paper/2102.00515/) (2021)
- [Sequential Random Network for Fine-grained Image Classification](https://mlsanity.com/paper/2103.07230/) (2021)
- [ProgressiveSpinalNet architecture for FC layers](https://mlsanity.com/paper/2103.11373/) (2021)
- [Self-Evolutionary Clustering](https://mlsanity.com/paper/2202.10505/) (2022)

## Abstract (arXiv)

Deep neural networks (DNNs) have achieved the state of the art performance in numerous fields. However, DNNs need high computation times, and people always expect better performance in a lower computation. Therefore, we study the human somatosensory system and design a neural network (SpinalNet) to achieve higher accuracy with fewer computations. Hidden layers in traditional NNs receive inputs in the previous layer, apply activation function, and then transfer the outcomes to the next layer. In the proposed SpinalNet, each layer is split into three splits: 1) input split, 2) intermediate split, and 3) output split. Input split of each layer receives a part of the inputs. The intermediate split of each layer receives outputs of the intermediate split of the previous layer and outputs of the input split of the current layer. The number of incoming weights becomes significantly lower than traditional DNNs. The SpinalNet can also be used as the fully connected or classification layer of DNN and supports both traditional learning and transfer learning. We observe significant error reductions with lower computational costs in most of the DNNs. Traditional learning on the VGG-5 network with SpinalNet classification layers provided the state-of-the-art (SOTA) performance on QMNIST, Kuzushiji-MNIST, EMNIST (Letters, Digits, and Balanced) datasets. Traditional learning with ImageNet pre-trained initial weights and SpinalNet classification layers provided the SOTA performance on STL-10, Fruits 360, Bird225, and Caltech-101 datasets. The scripts of the proposed SpinalNet are available at the following link: https://github.com/dipuk0506/SpinalNet