# MARTI-4: new model of human brain, considering neocortex and basal ganglia -- learns to play Atari game by reinforcement learning on a single CPU

arXiv: https://arxiv.org/abs/2209.02387 · HTML: https://mlsanity.com/paper/2209.02387/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 19 references that are in the map.
- First posted on arXiv 18 Aug 2022.

## Builds on (most cited first, 6 of 6)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [The Option-Critic Architecture](https://mlsanity.com/paper/1609.05140/) (2016)
- [FeUdal Networks for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1703.01161/) (2017)
- [Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex](https://mlsanity.com/paper/1511.00083/) (2015)
- [Feynman Machine: The Universal Dynamical Systems Computer](https://mlsanity.com/paper/1609.03971/) (2016)

## Abstract (arXiv)

We present Deep Control - new ML architecture of cortico-striatal brain circuits, which use whole cortical column as a structural element, instead of a singe neuron. Based on this architecture, we present MARTI - new model of human brain, considering neocortex and basal ganglia. This model is de-signed to implement expedient behavior and is capable to learn and achieve goals in unknown environments. We introduce a novel surprise feeling mechanism, that significantly improves reinforcement learning process through inner rewards. We use OpenAI Gym environment to demonstrate MARTI learning on a single CPU just in several hours.