mlsanityData to 14 Oct 2026

Deep reinforcement learning with time-scale invariant memory

arXiv 2412.15292 ↗First posted 19 Dec 2024cs.AI cs.LGcited by 1

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

The ability to estimate temporal relationships is critical for both animals and artificial agents. Cognitive science and neuroscience provide remarkable insights into behavioral and neural aspects of temporal credit assignment. In particular, scale invariance of learning dynamics, observed in behavior and supported by neural data, is one of the key principles that governs animal perception: proportional rescaling of temporal relationships does not alter the overall learning efficiency. Here we integrate a computational neuroscience model of scale invariant memory into deep reinforcement learning (RL) agents. We first provide a theoretical analysis and then demonstrate through experiments that such agents can learn robustly across a wide range of temporal scales, unlike agents built with commonly used recurrent memory architectures such as LSTM. This result illustrates that incorporating computational principles from neuroscience and cognitive science into deep neural networks can enhance adaptability to complex temporal dynamics, mirroring some of the core properties of human learning.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 8 of its 62 references that are in the map. First posted on arXiv 19 Dec 2024.

Builds on

8 of 62 references.

References · 8

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  4. 2021Relating transformers to models and neural representations of the hippocampal formationcited by 62
  5. 2019Interval timing in deep reinforcement learning agentscited by 7
  6. 2021DeepSITH: Efficient Learning via Decomposition of What and When Across Time Scalescited by 5
  7. 2021A deep convolutional neural network that is invariant to time rescalingcited by 5
  8. 2018Estimating scale-invariant future in continuous timecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Dynamical Priors as a Training Objective in Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Emergent time-keeping mechanisms in a deep reinforcement learning agent performing an interval timing task
  2. 2021A brain basis of dynamical intelligence for AI and computational neuroscience
  3. 2025Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environments
  4. 2026Task-Induced Representational Invariances Depend on Learning Objective in Deep RL
  5. 2025State Space Models Naturally Produce Time Cell and Oscillatory Behaviors and Scale to Abstract Cognitive Functions
  6. 2025From Memories to Maps: Mechanisms of In-Context Reinforcement Learning in Transformers
  7. 2025Quantifying Memory Use in Reinforcement Learning with Temporal Range
  8. 2025What Neuroscience Can Teach AI About Learning in Continuously Changing Environments