# A New Type of Neurons for Machine Learning

arXiv: https://arxiv.org/abs/1704.08362 · HTML: https://mlsanity.com/paper/1704.08362/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2017.

## Built on by (most cited first, 16 of 16)

- [Quadratic Residual Networks: A New Class of Neural Networks for Solving Forward and Inverse Problems in Physics Involving PDEs](https://mlsanity.com/paper/2101.08366/) (2021)
- [QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration](https://mlsanity.com/paper/2204.01701/) (2022)
- [Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks](https://mlsanity.com/paper/2301.09245/) (2023)
- [One Neuron Saved Is One Neuron Earned: On Parametric Efficiency of Quadratic Networks](https://mlsanity.com/paper/2303.06316/) (2023)
- [Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2204.01707/) (2022)
- [QuadraNet: Improving High-Order Neural Interaction Efficiency with Hardware-Aware Quadratic Neural Networks](https://mlsanity.com/paper/2311.17956/) (2023)
- [No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks](https://mlsanity.com/paper/2405.02369/) (2024)
- [Revisiting Feature Interactions from the Perspective of Quadratic Neural Networks for Click-through Rate Prediction](https://mlsanity.com/paper/2505.17999/) (2025)
- [A Quadratic Actor Network for Model-Free Reinforcement Learning](https://mlsanity.com/paper/2103.06617/) (2021)
- [NEWRON: A New Generalization of the Artificial Neuron to Enhance the Interpretability of Neural Networks](https://mlsanity.com/paper/2110.02775/) (2021)
- [Cloud-RAIN: Point Cloud Analysis with Reflectional Invariance](https://mlsanity.com/paper/2305.07814/) (2023)
- [Dendritic Integration Based Quadratic Neural Networks Outperform Traditional Aritificial Ones](https://mlsanity.com/paper/2307.13609/) (2023)
- [Efficient Vectorized Backpropagation Algorithms for Training Feedforward Networks Composed of Quadratic Neurons](https://mlsanity.com/paper/2310.02901/) (2023)
- [Retinal Vessel Segmentation via Neuron Programming](https://mlsanity.com/paper/2411.11110/) (2024)
- [NeuronSeek: On Stability and Expressivity of Task-driven Neurons](https://mlsanity.com/paper/2506.15715/) (2025)
- [TNLearn: An Open Source Python Package for Task-based Neurons](https://mlsanity.com/paper/2609.27564/) (2026)

## Abstract (arXiv)

In machine learning, the use of an artificial neural network is the mainstream approach. Such a network consists of layers of neurons. These neurons are of the same type characterized by the two features: (1) an inner product of an input vector and a matching weighting vector of trainable parameters and (2) a nonlinear excitation function. Here we investigate the possibility of replacing the inner product with a quadratic function of the input vector, thereby upgrading the 1st order neuron to the 2nd order neuron, empowering individual neurons, and facilitating the optimization of neural networks. Also, numerical examples are provided to illustrate the feasibility and merits of the 2nd order neurons. Finally, further topics are discussed.