# On learning spatial sequences with the movement of attention

arXiv: https://arxiv.org/abs/2311.06856 · HTML: https://mlsanity.com/paper/2311.06856/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 35 references that are in the map.
- First posted on arXiv 12 Nov 2023.

## Builds on (most cited first, 8 of 8)

- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [A comprehensive survey on point cloud registration](https://mlsanity.com/paper/2103.02690/) (2021)
- [VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem](https://mlsanity.com/paper/1701.08376/) (2017)
- [FutureMapping: The Computational Structure of Spatial AI Systems](https://mlsanity.com/paper/1803.11288/) (2018)
- [Space is a latent sequence: Structured sequence learning as a unified theory of representation in the hippocampus](https://mlsanity.com/paper/2212.01508/) (2022)
- [Formation of cell assemblies with iterative winners-take-all computation and excitation-inhibition balance](https://mlsanity.com/paper/2108.00706/) (2021)

## Abstract (arXiv)

In this paper we start with a simple question, how is it possible that humans can recognize different movements over skin with only a prior visual experience of them? Or in general, what is the representation of spatial sequences that are invariant to scale, rotation, and translation across different modalities? To answer, we rethink the mathematical representation of spatial sequences, argue against the minimum description length principle, and focus on the movements of attention. We advance the idea that spatial sequences must be represented on different levels of abstraction, this adds redundancy but is necessary for recognition and generalization. To address the open question of how these abstractions are formed we propose two hypotheses: the first invites exploring selectionism learning, instead of finding parameters in some models; the second proposes to find new data structures, not neural network architectures, to efficiently store and operate over redundant features to be further selected. Movements of attention are central to human cognition and lessons should be applied to new better learning algorithms.