# APTx Neuron: A Unified Trainable Neuron Architecture Integrating Activation and Computation

arXiv: https://arxiv.org/abs/2507.14270 · HTML: https://mlsanity.com/paper/2507.14270/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 17 references that are in the map.
- First posted on arXiv 18 Jul 2025.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [A survey on modern trainable activation functions](https://mlsanity.com/paper/2005.00817/) (2020)
- [Review and Comparison of Commonly Used Activation Functions for Deep Neural Networks](https://mlsanity.com/paper/2010.09458/) (2020)
- [APTx: better activation function than MISH, SWISH, and ReLU's variants used in deep learning](https://mlsanity.com/paper/2209.06119/) (2022)

## Abstract (arXiv)

We propose the APTx Neuron, a novel, unified neural computation unit that integrates non-linear activation and linear transformation into a single trainable expression. The APTx Neuron is derived from the APTx activation function, thereby eliminating the need for separate activation layers and making the architecture both optimization-efficient and elegant. The proposed neuron follows the functional form y = sumi=1n ((αi + tanh(βi xi)) · γi xi) + δ, where all parameters αi, βi, γi, and δ are trainable. We validate our APTx Neuron-based architecture on the MNIST dataset, achieving up to 96.69% test accuracy within 11 epochs using approximately 332K trainable parameters. The results highlight the superior expressiveness and training efficiency of the APTx Neuron compared to traditional neurons, pointing toward a new paradigm in unified neuron design and the architectures built upon it. Source code is available at https://github.com/mr-ravin/aptxneuron.