Temporal Chunking Enhances Recognition of Implicit Sequential Patterns
Not on the map. Why?
Abstract (arXiv)
In this pilot study, we propose a neuro-inspired approach that compresses temporal sequences into context-tagged chunks, where each tag represents a recurring structural unit or``community'' in the sequence. These tags are generated during an offline sleep phase and serve as compact references to past experience, allowing the learner to incorporate information beyond its immediate input range. We evaluate this idea in a controlled synthetic environment designed to reveal the limitations of traditional neural network based sequence learners, such as recurrent neural networks (RNNs), when facing temporal patterns on multiple timescales. We evaluate this idea in a controlled synthetic environment designed to reveal the limitations of traditional neural network based sequence learners, such as recurrent neural networks (RNNs), when facing temporal patterns on multiple timescales. Our results, while preliminary, suggest that temporal chunking can significantly enhance learning efficiency under resource constrained settings. A small-scale human pilot study using a Serial Reaction Time Task further motivates the idea of structural abstraction. Although limited to synthetic tasks, this work serves as an early proof-of-concept, with initial evidence that learned context tags can transfer across related task, offering potential for future applications in transfer learning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 30 references that are in the map. First posted on arXiv 31 May 2025.
Builds on
13 of 30 references.
References · 13
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
- 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
- 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
- 2014Neural Turing Machinescited by 644
- 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 514
- 2015Unitary Evolution Recurrent Neural Networkscited by 294
- 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 275
- 2019AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networkscited by 112
- 2016Hierarchical Multiscale Recurrent Neural Networkscited by 104
- 2017Dilated Recurrent Neural Networkscited by 83
- 2014A Clockwork RNNcited by 72
1 more
Built on by
Not cited yet.