# Operational Neural Networks

arXiv: https://arxiv.org/abs/1902.11106 · HTML: https://mlsanity.com/paper/1902.11106/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2019.

## Built on by (most cited first, 29 of 29)

- [Real-Time Patient-Specific ECG Classification by 1D Self-Operational Neural Networks](https://mlsanity.com/paper/2110.02215/) (2021)
- [Real-Time Glaucoma Detection from Digital Fundus Images using Self-ONNs](https://mlsanity.com/paper/2109.13604/) (2021)
- [Robust Peak Detection for Holter ECGs by Self-Organized Operational Neural Networks](https://mlsanity.com/paper/2110.02381/) (2021)
- [Blind ECG Restoration by Operational Cycle-GANs](https://mlsanity.com/paper/2202.00589/) (2022)
- [Global ECG Classification by Self-Operational Neural Networks with Feature Injection](https://mlsanity.com/paper/2204.03768/) (2022)
- [Self-Organized Variational Autoencoders (Self-VAE) for Learned Image Compression](https://mlsanity.com/paper/2105.12107/) (2021)
- [BM3D vs 2-Layer ONN](https://mlsanity.com/paper/2103.03060/) (2021)
- [Super Neurons](https://mlsanity.com/paper/2109.01594/) (2021)
- [Augmented Bilinear Network for Incremental Multi-Stock Time-Series Classification](https://mlsanity.com/paper/2207.11577/) (2022)
- [Zero-Shot Motor Health Monitoring by Blind Domain Transition](https://mlsanity.com/paper/2212.06154/) (2022)
- [DroneNet: Crowd Density Estimation using Self-ONNs for Drones](https://mlsanity.com/paper/2211.07137/) (2022)
- [Blind Restoration of Real-World Audio by 1D Operational GANs](https://mlsanity.com/paper/2212.14618/) (2022)
- [Improved Active Fire Detection using Operational U-Nets](https://mlsanity.com/paper/2304.09721/) (2023)
- [Convolutional versus Self-Organized Operational Neural Networks for Real-World Blind Image Denoising](https://mlsanity.com/paper/2103.03070/) (2021)
- [2D Self-Organized ONN Model For Handwritten Text Recognition](https://mlsanity.com/paper/2207.08139/) (2022)
- [Operational Support Estimator Networks](https://mlsanity.com/paper/2307.06065/) (2023)
- [Real-Time Vibration-Based Bearing Fault Diagnosis Under Time-Varying Speed Conditions](https://mlsanity.com/paper/2311.18547/) (2023)
- [PAON: A New Neuron Model using Pad'e Approximants](https://mlsanity.com/paper/2403.11791/) (2024)
- [Self-DenseMobileNet: A Robust Framework for Lung Nodule Classification using Self-ONN and Stacking-based Meta-Classifier](https://mlsanity.com/paper/2410.12584/) (2024)
- [Thermal Image-based Fault Diagnosis in Induction Machines via Self-Organized Operational Neural Networks](https://mlsanity.com/paper/2412.05901/) (2024)
- [Image denoising by Super Neurons: Why go deep?](https://mlsanity.com/paper/2111.14948/) (2021)
- [Exploring Sound vs Vibration for Robust Fault Detection on Rotating Machinery](https://mlsanity.com/paper/2312.10742/) (2023)
- [Fusion of Quadratic Time-Frequency Analysis and Convolutional Neural Networks to Diagnose Bearing Faults Under Time-Varying Speeds](https://mlsanity.com/paper/2401.01172/) (2024)
- [Informatics & dairy industry coalition: AI trends and present challenges](https://mlsanity.com/paper/2406.12770/) (2024)
- [High-Quality and Full Bandwidth Seismic Signal Synthesis using Operational GANs](https://mlsanity.com/paper/2407.11040/) (2024)
- [BRSR-OpGAN: Blind Radar Signal Restoration using Operational Generative Adversarial Network](https://mlsanity.com/paper/2407.13949/) (2024)
- [Self-CephaloNet: A Two-stage Novel Framework using Operational Neural Network for Cephalometric Analysis](https://mlsanity.com/paper/2501.10984/) (2025)
- [Pad'e Approximant Neural Networks for Enhanced Electric Motor Fault Diagnosis Using Vibration and Acoustic Data](https://mlsanity.com/paper/2507.02599/) (2025)
- [Pad'e Neurons for Efficient Neural Models](https://mlsanity.com/paper/2601.04005/) (2026)

## Abstract (arXiv)

Feed-forward, fully-connected Artificial Neural Networks (ANNs) or the so-called Multi-Layer Perceptrons (MLPs) are well-known universal approximators. However, their learning performance varies significantly depending on the function or the solution space that they attempt to approximate. This is mainly because of their homogenous configuration based solely on the linear neuron model. Therefore, while they learn very well those problems with a monotonous, relatively simple and linearly separable solution space, they may entirely fail to do so when the solution space is highly nonlinear and complex. Sharing the same linear neuron model with two additional constraints (local connections and weight sharing), this is also true for the conventional Convolutional Neural Networks (CNNs) and, it is, therefore, not surprising that in many challenging problems only the deep CNNs with a massive complexity and depth can achieve the required diversity and the learning performance. In order to address this drawback and also to accomplish a more generalized model over the convolutional neurons, this study proposes a novel network model, called Operational Neural Networks (ONNs), which can be heterogeneous and encapsulate neurons with any set of operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. Finally, a novel training method is formulated to back-propagate the error through the operational layers of ONNs. Experimental results over highly challenging problems demonstrate the superior learning capabilities of ONNs even with few neurons and hidden layers.