# Super Neurons

arXiv: https://arxiv.org/abs/2109.01594 · HTML: https://mlsanity.com/paper/2109.01594/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 67 references that are in the map.
- First posted on arXiv 3 Aug 2021.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks](https://mlsanity.com/paper/1711.07064/) (2017)
- [Scale-recurrent Network for Deep Image Deblurring](https://mlsanity.com/paper/1802.01770/) (2018)
- [Learning a Convolutional Neural Network for Non-uniform Motion Blur Removal](https://mlsanity.com/paper/1503.00593/) (2015)
- [Fully Convolutional Instance-aware Semantic Segmentation](https://mlsanity.com/paper/1611.07709/) (2016)
- [From Motion Blur to Motion Flow: a Deep Learning Solution for Removing Heterogeneous Motion Blur](https://mlsanity.com/paper/1612.02583/) (2016)
- [Semantic Segmentation using Adversarial Networks](https://mlsanity.com/paper/1611.08408/) (2016)
- [A Neural Approach to Blind Motion Deblurring](https://mlsanity.com/paper/1603.04771/) (2016)
- [Learning to Deblur](https://mlsanity.com/paper/1406.7444/) (2014)
- [Non-Local Color Image Denoising with Convolutional Neural Networks](https://mlsanity.com/paper/1611.06757/) (2016)
- [Self-Organized Operational Neural Networks with Generative Neurons](https://mlsanity.com/paper/2004.11778/) (2020)
- [Operational Neural Networks](https://mlsanity.com/paper/1902.11106/) (2019)
- [Self-Organized Operational Neural Networks for Severe Image Restoration Problems](https://mlsanity.com/paper/2008.12894/) (2020)
- [On learning optimized reaction diffusion processes for effective image restoration](https://mlsanity.com/paper/1503.05768/) (2015)
- [FastONN -- Python based open-source GPU implementation for Operational Neural Networks](https://mlsanity.com/paper/2006.02267/) (2020)
- [Motion Deblurring in the Wild](https://mlsanity.com/paper/1701.01486/) (2017)
- [Deep Generative Filter for Motion Deblurring](https://mlsanity.com/paper/1709.03481/) (2017)
- [Heterogeneous Multilayer Generalized Operational Perceptron](https://mlsanity.com/paper/1804.05093/) (2018)
- [Exploiting Heterogeneity in Operational Neural Networks by Synaptic Plasticity](https://mlsanity.com/paper/2009.08934/) (2020)
- [Beyond Pixels: A Comprehensive Survey from Bottom-up to Semantic Image Segmentation and Cosegmentation](https://mlsanity.com/paper/1502.00717/) (2015)
- [Progressive Operational Perceptron with Memory](https://mlsanity.com/paper/1808.06377/) (2018)
- [Operational vs Convolutional Neural Networks for Image Denoising](https://mlsanity.com/paper/2009.00612/) (2020)

## Built on by (most cited first, 7 of 7)

- [Blind ECG Restoration by Operational Cycle-GANs](https://mlsanity.com/paper/2202.00589/) (2022)
- [Global ECG Classification by Self-Operational Neural Networks with Feature Injection](https://mlsanity.com/paper/2204.03768/) (2022)
- [Blind Restoration of Real-World Audio by 1D Operational GANs](https://mlsanity.com/paper/2212.14618/) (2022)
- [Operational Support Estimator Networks](https://mlsanity.com/paper/2307.06065/) (2023)
- [PAON: A New Neuron Model using Pad'e Approximants](https://mlsanity.com/paper/2403.11791/) (2024)
- [Exploring Sound vs Vibration for Robust Fault Detection on Rotating Machinery](https://mlsanity.com/paper/2312.10742/) (2023)
- [Pad'e Neurons for Efficient Neural Models](https://mlsanity.com/paper/2601.04005/) (2026)

## Abstract (arXiv)

Self-Organized Operational Neural Networks (Self-ONNs) have recently been proposed as new-generation neural network models with nonlinear learning units, i.e., the generative neurons that yield an elegant level of diversity; however, like its predecessor, conventional Convolutional Neural Networks (CNNs), they still have a common drawback: localized (fixed) kernel operations. This severely limits the receptive field and information flow between layers and thus brings the necessity for deep and complex models. It is highly desired to improve the receptive field size without increasing the kernel dimensions. This requires a significant upgrade over the generative neurons to achieve the non-localized kernel operations for each connection between consecutive layers. In this article, we present superior (generative) neuron models (or super neurons in short) that allow random or learnable kernel shifts and thus can increase the receptive field size of each connection. The kernel localization process varies among the two super-neuron models. The first model assumes randomly localized kernels within a range and the second one learns (optimizes) the kernel locations during training. An extensive set of comparative evaluations against conventional and deformable convolutional, along with the generative neurons demonstrates that super neurons can empower Self-ONNs to achieve a superior learning and generalization capability with a minimal computational complexity burden.