mlsanityData to 7 Oct 2026

Deep Learning 2.0: Artificial Neurons That Matter -- Reject Correlation, Embrace Orthogonality

arXiv 2411.08085 ↗First posted 12 Nov 2024cs.LG cs.CV math.GNcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce a yat-product-powered neural network, the Neural Matter Network (NMN), a breakthrough in deep learning that achieves non-linear pattern recognition without activation functions. Our key innovation relies on the yat-product and yat-product, which naturally induces non-linearity by projecting inputs into a pseudo-metric space, eliminating the need for traditional activation functions while maintaining only a softmax layer for final class probability distribution. This approach simplifies network architecture and provides unprecedented transparency into the network's decision-making process. Our comprehensive empirical evaluation across different datasets demonstrates that NMN consistently outperforms traditional MLPs. The results challenge the assumption that separate activation functions are necessary for effective deep-learning models. The implications of this work extend beyond immediate architectural benefits, by eliminating intermediate activation functions while preserving non-linear capabilities, yat-MLP establishes a new paradigm for neural network design that combines simplicity with effectiveness. Most importantly, our approach provides unprecedented insights into the traditionally opaque "black-box" nature of neural networks, offering a clearer understanding of how these models process and classify information.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 29 references that are in the map. First posted on arXiv 12 Nov 2024.

Builds on

11 of 29 references.

References · 11

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  3. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  4. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  5. 2016Geometric deep learning: going beyond Euclidean datacited by 1,042
  6. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  7. 2021Pay Attention to MLPscited by 367
  8. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 262
  9. 2022Annotated History of Modern AI and Deep Learningcited by 20
  10. 2023The Grand Illusion: The Myth of Software Portability and Implications for ML Progresscited by 3
  11. 2024SimO Loss: Anchor-Free Contrastive Loss for Fine-Grained Supervised Contrastive Learningcited by 1

Built on by

Not cited yet.