# Scalable Machines with Intrinsic Higher Mental-State Dynamics

arXiv: https://arxiv.org/abs/2603.13453 · HTML: https://mlsanity.com/paper/2603.13453/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 80 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [DeepSeek LLM: Scaling Open-Source Language Models with Longtermism](https://mlsanity.com/paper/2401.02954/) (2024)
- [Dendritic cortical microcircuits approximate the backpropagation algorithm](https://mlsanity.com/paper/1810.11393/) (2018)
- [Towards deep learning with segregated dendrites](https://mlsanity.com/paper/1610.00161/) (2016)
- [Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense](https://mlsanity.com/paper/2004.09044/) (2020)
- [The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/2109.02869/) (2021)
- [Local plasticity rules can learn deep representations using self-supervised contrastive predictions](https://mlsanity.com/paper/2010.08262/) (2020)
- [Single-phase deep learning in cortico-cortical networks](https://mlsanity.com/paper/2206.11769/) (2022)
- [The Dynamics of Functional Brain Networks: Integrated Network States during Cognitive Function](https://mlsanity.com/paper/1511.02976/) (2015)
- [Leveraging dendritic properties to advance machine learning and neuro-inspired computing](https://mlsanity.com/paper/2306.08007/) (2023)
- [A comparison of partial information decompositions using data from real and simulated layer 5b pyramidal cells](https://mlsanity.com/paper/2206.06456/) (2022)
- [Unlocking the potential of two-point cells for energy-efficient and resilient training of deep nets](https://mlsanity.com/paper/2211.01950/) (2022)
- [Cooperation Is All You Need](https://mlsanity.com/paper/2305.10449/) (2023)
- [Two-compartment neuronal spiking model expressing brain-state specific apical-amplification, -isolation and -drive regimes](https://mlsanity.com/paper/2311.06074/) (2023)
- [An Overlooked Role of Context-Sensitive Dendrites](https://mlsanity.com/paper/2408.11019/) (2024)
- [Beyond Attention: Toward Machines with Intrinsic Higher Mental States](https://mlsanity.com/paper/2505.06257/) (2025)

## Built on by (most cited first, 2 of 2)

- [Programs-of-Layers in LLMs through the Lens of Cortical Areas](https://mlsanity.com/paper/2609.31360/) (2026)
- [A Cellular Doctrine of Morality: Intrinsic Active Precision and the Mind-Reality Overload Dilemma](https://mlsanity.com/paper/2605.01376/) (2026)

## Abstract (arXiv)

Drawing on recent breakthroughs in cellular neurobiology and detailed biophysical modeling linking neocortical pyramidal neurons to distinct mental-state regimes, this work introduces a mathematically grounded formulation showing how models (e.g., Transformers) can implement computational principles underlying awake imaginative thought to pre-select relevant information before attention is applied via triadic modulation loops among queries (Q), keys (K), and values (V).~Scalability experiments on ImageNet-1K, benchmarked against a standard Vision Transformer (ViT), demonstrate significantly faster learning with reduced computational demand (fewer heads, layers, and tokens), consistent with our prior findings in reinforcement learning and language modeling. The approach operates at approximately O(N) complexity with respect to the number of input tokens N.