# Self-Organized Operational Neural Networks with Generative Neurons

arXiv: https://arxiv.org/abs/2004.11778 · HTML: https://mlsanity.com/paper/2004.11778/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2020.

## Built on by (most cited first, 30 of 38)

- [OSegNet: Operational Segmentation Network for COVID-19 Detection using Chest X-ray Images](https://mlsanity.com/paper/2202.10185/) (2022)
- [COVID-19 Infection Localization and Severity Grading from Chest X-ray Images](https://mlsanity.com/paper/2103.07985/) (2021)
- [Real-Time Patient-Specific ECG Classification by 1D Self-Operational Neural Networks](https://mlsanity.com/paper/2110.02215/) (2021)
- [Real-Time Glaucoma Detection from Digital Fundus Images using Self-ONNs](https://mlsanity.com/paper/2109.13604/) (2021)
- [Robust Peak Detection for Holter ECGs by Self-Organized Operational Neural Networks](https://mlsanity.com/paper/2110.02381/) (2021)
- [Blind ECG Restoration by Operational Cycle-GANs](https://mlsanity.com/paper/2202.00589/) (2022)
- [Global ECG Classification by Self-Operational Neural Networks with Feature Injection](https://mlsanity.com/paper/2204.03768/) (2022)
- [Self-Organized Variational Autoencoders (Self-VAE) for Learned Image Compression](https://mlsanity.com/paper/2105.12107/) (2021)
- [BM3D vs 2-Layer ONN](https://mlsanity.com/paper/2103.03060/) (2021)
- [Super Neurons](https://mlsanity.com/paper/2109.01594/) (2021)
- [SRL-SOA: Self-Representation Learning with Sparse 1D-Operational Autoencoder for Hyperspectral Image Band Selection](https://mlsanity.com/paper/2202.09918/) (2022)
- [Zero-Shot Motor Health Monitoring by Blind Domain Transition](https://mlsanity.com/paper/2212.06154/) (2022)
- [Split Conformal Prediction in the Function Space with Neural Operators](https://mlsanity.com/paper/2509.04623/) (2025)
- [Blind Restoration of Real-World Audio by 1D Operational GANs](https://mlsanity.com/paper/2212.14618/) (2022)
- [Improved Active Fire Detection using Operational U-Nets](https://mlsanity.com/paper/2304.09721/) (2023)
- [Deep Learning-Driven Segmentation of Ischemic Stroke Lesions Using Multi-Channel MRI](https://mlsanity.com/paper/2501.02287/) (2025)
- [Convolutional versus Self-Organized Operational Neural Networks for Real-World Blind Image Denoising](https://mlsanity.com/paper/2103.03070/) (2021)
- [2D Self-Organized ONN Model For Handwritten Text Recognition](https://mlsanity.com/paper/2207.08139/) (2022)
- [R2C-GAN: Restore-to-Classify Generative Adversarial Networks for Blind X-Ray Restoration and COVID-19 Classification](https://mlsanity.com/paper/2209.14770/) (2022)
- [Operational Support Estimator Networks](https://mlsanity.com/paper/2307.06065/) (2023)
- [Real-Time Vibration-Based Bearing Fault Diagnosis Under Time-Varying Speed Conditions](https://mlsanity.com/paper/2311.18547/) (2023)
- [Joint End-to-End Image Compression and Denoising: Leveraging Contrastive Learning and Multi-Scale Self-ONNs](https://mlsanity.com/paper/2402.05582/) (2024)
- [PAON: A New Neuron Model using Pad'e Approximants](https://mlsanity.com/paper/2403.11791/) (2024)
- [Machine-agnostic Automated Lumbar MRI Segmentation using a Cascaded Model Based on Generative Neurons](https://mlsanity.com/paper/2411.15656/) (2024)
- [Thermal Image-based Fault Diagnosis in Induction Machines via Self-Organized Operational Neural Networks](https://mlsanity.com/paper/2412.05901/) (2024)
- [Expert Operational GANS: Towards Real-Color Underwater Image Restoration](https://mlsanity.com/paper/2507.11562/) (2025)
- [CASR-Net: An Image Processing-focused Deep Learning-based Coronary Artery Segmentation and Refinement Network for X-ray Coronary Angiogram](https://mlsanity.com/paper/2510.27315/) (2025)
- [Image denoising by Super Neurons: Why go deep?](https://mlsanity.com/paper/2111.14948/) (2021)
- [Exploring Sound vs Vibration for Robust Fault Detection on Rotating Machinery](https://mlsanity.com/paper/2312.10742/) (2023)
- [Fusion of Quadratic Time-Frequency Analysis and Convolutional Neural Networks to Diagnose Bearing Faults Under Time-Varying Speeds](https://mlsanity.com/paper/2401.01172/) (2024)

## Abstract (arXiv)

Operational Neural Networks (ONNs) have recently been proposed to address the well-known limitations and drawbacks of conventional Convolutional Neural Networks (CNNs) such as network homogeneity with the sole linear neuron model. ONNs are heterogenous networks with a generalized neuron model that can encapsulate any set of non-linear operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. However, Greedy Iterative Search (GIS) method, which is the search method used to find optimal operators in ONNs takes many training sessions to find a single operator set per layer. This is not only computationally demanding, but the network heterogeneity is also limited since the same set of operators will then be used for all neurons in each layer. Moreover, the performance of ONNs directly depends on the operator set library used, which introduces a certain risk of performance degradation especially when the optimal operator set required for a particular task is missing from the library. In order to address these issues and achieve an ultimate heterogeneity level to boost the network diversity along with computational efficiency, in this study we propose Self-organized ONNs (Self-ONNs) with generative neurons that have the ability to adapt (optimize) the nodal operator of each connection during the training process. Therefore, Self-ONNs can have an utmost heterogeneity level required by the learning problem at hand. Moreover, this ability voids the need of having a fixed operator set library and the prior operator search within the library in order to find the best possible set of operators. We further formulate the training method to back-propagate the error through the operational layers of Self-ONNs.