mlsanityData to 7 Oct 2026

Vector Quantization in the Brain: Grid-like Codes in World Models

arXiv 2510.16039 ↗First posted 16 Oct 2025cs.LG cs.AIcited by 0

On the map: World models → Latent predictive world models

Abstract (arXiv)

We propose Grid-like Code Quantization (GCQ), a brain-inspired method for compressing observation-action sequences into discrete representations using grid-like patterns in attractor dynamics. Unlike conventional vector quantization approaches that operate on static inputs, GCQ performs spatiotemporal compression through an action-conditioned codebook, where codewords are derived from continuous attractor neural networks and dynamically selected based on actions. This enables GCQ to jointly compress space and time, serving as a unified world model. The resulting representation supports long-horizon prediction, goal-directed planning, and inverse modeling. Experiments across diverse tasks demonstrate GCQ's effectiveness in compact encoding and downstream performance. Our work offers both a computational tool for efficient sequence modeling and a theoretical perspective on the formation of grid-like codes in neural systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 24 of its 61 references that are in the map. First posted on arXiv 16 Oct 2025.

Builds on

24 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018Learning Latent Dynamics for Planning from Pixels12 Nov

Other references · 23

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2017Neural Discrete Representation Learningcited by 4,650
  4. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  6. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  7. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  8. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  9. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  10. 2024Navigation World Modelscited by 262
  11. 2023Language Modeling Is Compressioncited by 225
  12. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
11 more
  1. 2023Learning to Act without Actionscited by 116
  2. 2019On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Datasetcited by 100
  3. 2018Theory and Experiments on Vector Quantized Autoencoderscited by 55
  4. 2018Emergence of grid-like representations by training recurrent neural networks to perform spatial localizationcited by 52
  5. 2018Disentangling Factors of Variation with Cycle-Consistent Variational Auto-Encoderscited by 49
  6. 2024Mastering Memory Tasks with World Modelscited by 38
  7. 2023Facing Off World Model Backbones: RNNs, Transformers, and S4cited by 31
  8. 2022Disentanglement with Biological Constraints: A Theory of Functional Cell Typescited by 30
  9. 2021Variable-rate discrete representation learningcited by 24
  10. 2018Generalisation of structural knowledge in the hippocampal-entorhinal systemcited by 8
  11. 2025Predictive Learning in Energy-based Models with Attractor Structurescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Grid Cell-Inspired Structured Vector Algebra for Cognitive Maps
  2. 2026The Global Neural World Model: Spatially Grounded Discrete Topologies for Action-Conditioned Planning
  3. 2026QVLA: Not All Channels Are Equal in Vision-Language-Action Model's Quantization
  4. 2026Geometric Priors for Generalizable World Models via Vector Symbolic Architecture
  5. 2022Active Predictive Coding: A Unified Neural Framework for Learning Hierarchical World Models for Perception and Planning
  6. 2026SuperLocalMemory V3.3: The Living Brain -- Biologically-Inspired Forgetting, Cognitive Quantization, and Multi-Channel Retrieval for Zero-LLM Agent Memory Systems
  7. 2026Code World Model: Coding Agent as World Brain
  8. 2026Spatial Strategies, Not Actions: Vector-Quantized Geodesics as Tools for LLM-Driven Agents