mlsanityData to 7 Oct 2026

How Do Artificial Intelligences Think? The Three Mathematico-Cognitive Factors of Categorical Segmentation Operated by Synthetic Neurons

arXiv 2501.06196 ↗First posted 26 Dec 2024q-bio.NC cs.AI cs.NEcited by 3

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

How do the synthetic neurons in language models create "thought categories" to segment and analyze their informational environment? What are the cognitive characteristics, at the very level of formal neurons, of this artificial categorical thought? Based on the mathematical nature of algebraic operations inherent to neuronal aggregation functions, we attempt to identify mathematico-cognitive factors that genetically shape the categorical reconstruction of the informational world faced by artificial cognition. This study explores these concepts through the notions of priming, attention, and categorical phasing.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 28 of its 82 references that are in the map. First posted on arXiv 26 Dec 2024.

Builds on

28 of 82 references.

References · 28

  1. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  2. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  3. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 572
  4. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 542
  5. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452
  6. 2023Explainability for Large Language Models: A Surveycited by 331
  7. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  8. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  9. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 168
  10. 2023Vision-Language Models as Success Detectorscited by 138
  11. 2020Generating Fact Checking Explanationscited by 132
  12. 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
16 more
  1. 2021Abstraction and Analogy-Making in Artificial Intelligencecited by 103
  2. 2021"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classificationcited by 66
  3. 2024ReLU2 Wins: Discovering Efficient Activation Functions for Sparse LLMscited by 66
  4. 2022Discovering Latent Concepts Learned in BERTcited by 56
  5. 2024BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcutscited by 36
  6. 2021Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMTcited by 21
  7. 2022On the Transformation of Latent Space in Fine-Tuned NLP Modelscited by 19
  8. 2023Can LLMs facilitate interpretation of pre-trained language models?cited by 15
  9. 2024The Pursuit of Fairness in Artificial Intelligence Models: A Surveycited by 13
  10. 2023Driving through the Concept Gridlock: Unraveling Explainability Bottlenecks in Automated Drivingcited by 9
  11. 2023Evaluating Neuron Interpretation Methods of NLP Modelscited by 8
  12. 2024A call for embodied AIcited by 7
  13. 2024Psychology of Artificial Intelligence: Epistemological Markers of the Cognitive Analysis of Neural Networkscited by 5
  14. 2024A hierarchical Bayesian model for syntactic primingcited by 4
  15. 2024The need of a self for self-driving cars a theoretical model applying homeostasis to self drivingcited by 1
  16. 2024A Neural Network Alternative to Tree-based Modelscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Synthetic Categorical Restructuring large Or How AIs Gradually Extract Efficient Regularities from Their Experience of the Worldcited by 2
  2. 2025Intra-neuronal attention within language models Relationships between activation and semanticscited by 0
  3. 2025Polysemy of Synthetic Neurons Towards a New Type of Explanatory Categorical Vector Spacescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025On Representational Dissociation of Language and Arithmetic in Large Language Models
  2. 2026Abstract representational geometry supports inference in large language models
  3. 2026Epistemology of Generative AI: The Geometry of Knowing
  4. 2025Visual Categorization Across Minds and Models: Cognitive Analysis of Human Labeling and Neuro-Symbolic Integration
  5. 2022A World-Self Model Towards Understanding Intelligence
  6. 2026Modular Cognitive Architecture Emerges in Large Language Models
  7. 2024A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debates
  8. 2025Language and Thought: The View from LLMs