mlsanityData to 14 Oct 2026

Polysemy of Synthetic Neurons Towards a New Type of Explanatory Categorical Vector Spaces

arXiv 2505.07831 ↗First posted 30 Apr 2025cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The polysemantic nature of synthetic neurons in artificial intelligence language models is currently understood as the result of a necessary superposition of distributed features within the latent space. We propose an alternative approach, geometrically defining a neuron in layer n as a categorical vector space with a non-orthogonal basis, composed of categorical sub-dimensions extracted from preceding neurons in layer n-1. This categorical vector space is structured by the activation space of each neuron and enables, via an intra-neuronal attention process, the identification and utilization of a critical categorical zone for the efficiency of the language model - more homogeneous and located at the intersection of these different categorical sub-dimensions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 62 of its 197 references that are in the map. First posted on arXiv 30 Apr 2025.

Builds on

62 of 197 references.

References · 62

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  8. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  9. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  10. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  11. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  12. 2020Reformer: The Efficient Transformercited by 1,668
38 more
  1. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  2. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  3. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
  4. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  5. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  6. 2019Are Sixteen Heads Really Better than One?cited by 816
  7. 2020Efficient Transformers: A Surveycited by 790
  8. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  9. 2024Scaling and evaluating sparse autoencoderscited by 553
  10. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  11. 2020Hopfield Networks is All You Needcited by 475
  12. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  13. 2023Orthogonal Subspace Learning for Language Model Continual Learningcited by 312
  14. 2024Massive Activations in Large Language Modelscited by 294
  15. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  16. 2020Evaluating and Aggregating Feature-based Model Explanationscited by 188
  17. 2020Generating Fact Checking Explanationscited by 132
  18. 2023A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimationcited by 109
  19. 2021Abstraction and Analogy-Making in Artificial Intelligencecited by 103
  20. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  21. 2017Machine learning for neural decodingcited by 45
  22. 2024BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcutscited by 37
  23. 2024Mathematical Models of Computation in Superpositioncited by 35
  24. 2024The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Unitscited by 34
  25. 2024Sharing Matters: Analysing Neurons Across Languages and Tasks in LLMscited by 33
  26. 2023Disentangling Neuron Representations with Concept Vectorscited by 28
  27. 2024Householder Pseudo-Rotation: A Novel Approach to Activation Editing in LLMs with Direction-Magnitude Perspectivecited by 23
  28. 2024Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Modelscited by 23
  29. 2021Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMTcited by 21
  30. 2024On the Complexity of Neural Computation in Superpositioncited by 21
  31. 2022On the Transformation of Latent Space in Fine-Tuned NLP Modelscited by 19
  32. 2023Can LLMs facilitate interpretation of pre-trained language models?cited by 16
  33. 2022Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Modelscited by 14
  34. 2024Can A Cognitive Architecture Fundamentally Enhance LLMs? Or Vice Versa?cited by 9
  35. 2024Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformerscited by 9
  36. 2025A PennyLane-Centric Dataset to Enhance LLM-based Quantum Code Generation using RAGcited by 9
  37. 2022Neural Network based Formation of Cognitive Maps of Semantic Spaces and the Emergence of Abstract Conceptscited by 8
  38. 2023Evaluating Neuron Interpretation Methods of NLP Modelscited by 8

50 most cited of 62 All 62 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Models
  2. 2024Neuropsychology and Explainability of AI: A Distributional Approach to the Relationship Between Activation Similarity of Neural Categories in Synthetic Cognition
  3. 2026NeuronScope: A Multi-Agent Framework for Explaining Polysemantic Neurons in Language Models
  4. 2025Disentangling Polysemantic Neurons with a Null-Calibrated Polysemanticity Index and Causal Patch Interventions
  5. 2026Neural networks are more modular than single neurons suggest
  6. 2026The Topological Dual of a Dataset: A Logic-to-Topology Encoding for AlphaGeometry-Style Data
  7. 2025Evaluating Sparse Autoencoders for Monosemantic Representation
  8. 2022Interpreting Neural Networks through the Polytope Lens