# How Do Artificial Intelligences Think? The Three Mathematico-Cognitive Factors of Categorical Segmentation Operated by Synthetic Neurons

arXiv: https://arxiv.org/abs/2501.06196 · HTML: https://mlsanity.com/paper/2501.06196/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 28 of its 82 references that are in the map.
- First posted on arXiv 26 Dec 2024.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 28 of 28)

- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Gradient Descent Finds Global Minima of Deep Neural Networks](https://mlsanity.com/paper/1811.03804/) (2018)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [Large Language Models Struggle to Learn Long-Tail Knowledge](https://mlsanity.com/paper/2211.08411/) (2022)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [A Survey of the State of Explainable AI for Natural Language Processing](https://mlsanity.com/paper/2010.00711/) (2020)
- [Sources of Hallucination by Large Language Models on Inference Tasks](https://mlsanity.com/paper/2305.14552/) (2023)
- [Vision-Language Models as Success Detectors](https://mlsanity.com/paper/2303.07280/) (2023)
- [Generating Fact Checking Explanations](https://mlsanity.com/paper/2004.05773/) (2020)
- [What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models](https://mlsanity.com/paper/1812.09355/) (2018)
- [Abstraction and Analogy-Making in Artificial Intelligence](https://mlsanity.com/paper/2102.10717/) (2021)
- [ReLU2 Wins: Discovering Efficient Activation Functions for Sparse LLMs](https://mlsanity.com/paper/2402.03804/) (2024)
- ["Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classification](https://mlsanity.com/paper/2111.07367/) (2021)
- [Discovering Latent Concepts Learned in BERT](https://mlsanity.com/paper/2205.07237/) (2022)
- [BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcuts](https://mlsanity.com/paper/2402.12240/) (2024)
- [Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMT](https://mlsanity.com/paper/2109.01396/) (2021)
- [On the Transformation of Latent Space in Fine-Tuned NLP Models](https://mlsanity.com/paper/2210.12696/) (2022)
- [Can LLMs facilitate interpretation of pre-trained language models?](https://mlsanity.com/paper/2305.13386/) (2023)
- [The Pursuit of Fairness in Artificial Intelligence Models: A Survey](https://mlsanity.com/paper/2403.17333/) (2024)
- [Driving through the Concept Gridlock: Unraveling Explainability Bottlenecks in Automated Driving](https://mlsanity.com/paper/2310.16639/) (2023)
- [Evaluating Neuron Interpretation Methods of NLP Models](https://mlsanity.com/paper/2301.12608/) (2023)
- [A call for embodied AI](https://mlsanity.com/paper/2402.03824/) (2024)
- [Psychology of Artificial Intelligence: Epistemological Markers of the Cognitive Analysis of Neural Networks](https://mlsanity.com/paper/2407.09563/) (2024)
- [A hierarchical Bayesian model for syntactic priming](https://mlsanity.com/paper/2405.15964/) (2024)
- [The need of a self for self-driving cars a theoretical model applying homeostasis to self driving](https://mlsanity.com/paper/2407.12795/) (2024)
- [A Neural Network Alternative to Tree-based Models](https://mlsanity.com/paper/2410.17758/) (2024)

## Built on by (most cited first, 3 of 3)

- [Synthetic Categorical Restructuring large Or How AIs Gradually Extract Efficient Regularities from Their Experience of the World](https://mlsanity.com/paper/2503.10643/) (2025)
- [Intra-neuronal attention within language models Relationships between activation and semantics](https://mlsanity.com/paper/2503.12992/) (2025)
- [Polysemy of Synthetic Neurons Towards a New Type of Explanatory Categorical Vector Spaces](https://mlsanity.com/paper/2505.07831/) (2025)

## Abstract (arXiv)

How do the synthetic neurons in language models create "thought categories" to segment and analyze their informational environment? What are the cognitive characteristics, at the very level of formal neurons, of this artificial categorical thought? Based on the mathematical nature of algebraic operations inherent to neuronal aggregation functions, we attempt to identify mathematico-cognitive factors that genetically shape the categorical reconstruction of the informational world faced by artificial cognition. This study explores these concepts through the notions of priming, attention, and categorical phasing.