# No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks

arXiv: https://arxiv.org/abs/2405.02369 · HTML: https://mlsanity.com/paper/2405.02369/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 60 references that are in the map.
- First posted on arXiv 3 May 2024.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [TabNet: Attentive Interpretable Tabular Learning](https://mlsanity.com/paper/1908.07442/) (2019)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [TabTransformer: Tabular Data Modeling Using Contextual Embeddings](https://mlsanity.com/paper/2012.06678/) (2020)
- [Explaining Neural Scaling Laws](https://mlsanity.com/paper/2102.06701/) (2021)
- [Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery](https://mlsanity.com/paper/1912.04825/) (2019)
- [Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution](https://mlsanity.com/paper/2210.08340/) (2022)
- [Symbolic Regression is NP-hard](https://mlsanity.com/paper/2207.01018/) (2022)
- [Deep Polynomial Neural Networks](https://mlsanity.com/paper/2006.13026/) (2020)
- [AutoML to Date and Beyond: Challenges and Opportunities](https://mlsanity.com/paper/2010.10777/) (2020)
- [DANets: Deep Abstract Networks for Tabular Data Classification and Regression](https://mlsanity.com/paper/2112.02962/) (2021)
- [Quadratic Residual Networks: A New Class of Neural Networks for Solving Forward and Inverse Problems in Physics Involving PDEs](https://mlsanity.com/paper/2101.08366/) (2021)
- [Non-linear Convolution Filters for CNN-based Learning](https://mlsanity.com/paper/1708.07038/) (2017)
- [Quadratic Autoencoder (Q-AE) for Low-dose CT Denoising](https://mlsanity.com/paper/1901.05593/) (2019)
- [Exhaustive Symbolic Regression](https://mlsanity.com/paper/2211.11461/) (2022)
- [Deep Learning and Symbolic Regression for Discovering Parametric Equations](https://mlsanity.com/paper/2207.00529/) (2022)
- [QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration](https://mlsanity.com/paper/2204.01701/) (2022)
- [On a Sparse Shortcut Topology of Artificial Neural Networks](https://mlsanity.com/paper/1811.09003/) (2018)
- [A Framework Based on Symbolic Regression Coupled with eXtended Physics-Informed Neural Networks for Gray-Box Learning of Equations of Motion from Data](https://mlsanity.com/paper/2305.10706/) (2023)
- [A New Type of Neurons for Machine Learning](https://mlsanity.com/paper/1704.08362/) (2017)
- [Multi-Grade Deep Learning](https://mlsanity.com/paper/2302.00150/) (2023)
- [Dendrite Net: A White-Box Module for Classification, Regression, and System Identification](https://mlsanity.com/paper/2004.03955/) (2020)
- [Attention-embedded Quadratic Network (Qttention) for Effective and Interpretable Bearing Fault Diagnosis](https://mlsanity.com/paper/2206.00390/) (2022)
- [TabNAS: Rejection Sampling for Neural Architecture Search on Tabular Datasets](https://mlsanity.com/paper/2204.07615/) (2022)
- [Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks](https://mlsanity.com/paper/2301.09245/) (2023)
- [One Neuron Saved Is One Neuron Earned: On Parametric Efficiency of Quadratic Networks](https://mlsanity.com/paper/2303.06316/) (2023)

## Built on by (most cited first, 3 of 3)

- [NeuronSeek: On Stability and Expressivity of Task-driven Neurons](https://mlsanity.com/paper/2506.15715/) (2025)
- [Effective Receptive Field Ordering Matters for Infrared Small Target Detection](https://mlsanity.com/paper/2607.23994/) (2026)
- [TNLearn: An Open Source Python Package for Task-based Neurons](https://mlsanity.com/paper/2609.27564/) (2026)

## Abstract (arXiv)

In the past decade, many successful networks are on novel architectures, which almost exclusively use the same type of neurons. Recently, more and more deep learning studies have been inspired by the idea of NeuroAI and the neuronal diversity observed in human brains, leading to the proposal of novel artificial neuron designs. Designing well-performing neurons represents a new dimension relative to designing well-performing neural architectures. Biologically, the brain does not rely on a single type of neuron that universally functions in all aspects. Instead, in our brain, neurons are often task-based. In this study, we address the following question: since the human brain is a task-based neuron user, can the artificial network design go from the task-based architecture design to the task-based neuron design? Since methodologically there are no one-size-fits-all neurons, given the same structure, task-based neurons can enhance the feature representation ability relative to the existing universal neurons due to the intrinsic inductive bias for the task. Specifically, we propose a two-step framework for prototyping task-based neurons. As the initial step, we evaluate the proposed framework using polynomials as base functions. Empirically, systematic experimental results on synthetic data, classic benchmarks, and real-world applications show that the proposed task-based neuron design is not only feasible but also delivers competitive performance over other state-of-the-art models.