mlsanityData to 14 Oct 2026

Scalable Machines with Intrinsic Higher Mental-State Dynamics

arXiv 2603.13453 ↗First posted 13 Mar 2026cs.LG cs.AIcited by 2

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Drawing on recent breakthroughs in cellular neurobiology and detailed biophysical modeling linking neocortical pyramidal neurons to distinct mental-state regimes, this work introduces a mathematically grounded formulation showing how models (e.g., Transformers) can implement computational principles underlying awake imaginative thought to pre-select relevant information before attention is applied via triadic modulation loops among queries (Q), keys (K), and values (V).~Scalability experiments on ImageNet-1K, benchmarked against a standard Vision Transformer (ViT), demonstrate significantly faster learning with reduced computational demand (fewer heads, layers, and tokens), consistent with our prior findings in reinforcement learning and language modeling. The approach operates at approximately O(N) complexity with respect to the number of input tokens N.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 80 references that are in the map. First posted on arXiv 13 Mar 2026.

Builds on

22 of 80 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  7. 2024VMamba: Visual State Space Modelcited by 969
  8. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  9. 2018Dendritic cortical microcircuits approximate the backpropagation algorithmcited by 97
  10. 2016Towards deep learning with segregated dendritescited by 71
  11. 2020Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sensecited by 62
  12. 2021The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learningcited by 50
10 more
  1. 2020Local plasticity rules can learn deep representations using self-supervised contrastive predictionscited by 47
  2. 2022Single-phase deep learning in cortico-cortical networkscited by 19
  3. 2015The Dynamics of Functional Brain Networks: Integrated Network States during Cognitive Functioncited by 17
  4. 2023Leveraging dendritic properties to advance machine learning and neuro-inspired computingcited by 6
  5. 2022A comparison of partial information decompositions using data from real and simulated layer 5b pyramidal cellscited by 5
  6. 2022Unlocking the potential of two-point cells for energy-efficient and resilient training of deep netscited by 5
  7. 2023Cooperation Is All You Needcited by 3
  8. 2023Two-compartment neuronal spiking model expressing brain-state specific apical-amplification, -isolation and -drive regimescited by 3
  9. 2024An Overlooked Role of Context-Sensitive Dendritescited by 3
  10. 2025Beyond Attention: Toward Machines with Intrinsic Higher Mental Statescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Programs-of-Layers in LLMs through the Lens of Cortical Areascited by 1
  2. 2026A Cellular Doctrine of Morality: Intrinsic Active Precision and the Mind-Reality Overload Dilemmacited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021A brain basis of dynamical intelligence for AI and computational neuroscience
  2. 2025Attention via Synaptic Plasticity is All You Need: A Biologically Inspired Spiking Neuromorphic Transformer
  3. 2025The brain-AI convergence: Predictive and generative world models for general-purpose computation
  4. 2025The Dragon Hatchling: The Missing Link between the Transformer and Models of the Brain
  5. 2026Self-Improving is Often Sudden: Enlightenment-style Finetuning for Large-Scale Models
  6. 2025From generative AI to the brain: five takeaways
  7. 2022Towards biologically plausible Dreaming and Planning in recurrent spiking networks
  8. 2025BriLLM: Brain-inspired Large Language Model