# Deep reinforcement learning with time-scale invariant memory

arXiv: https://arxiv.org/abs/2412.15292 · HTML: https://mlsanity.com/paper/2412.15292/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 62 references that are in the map.
- First posted on arXiv 19 Dec 2024.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Relating transformers to models and neural representations of the hippocampal formation](https://mlsanity.com/paper/2112.04035/) (2021)
- [Interval timing in deep reinforcement learning agents](https://mlsanity.com/paper/1905.13469/) (2019)
- [DeepSITH: Efficient Learning via Decomposition of What and When Across Time Scales](https://mlsanity.com/paper/2104.04646/) (2021)
- [A deep convolutional neural network that is invariant to time rescaling](https://mlsanity.com/paper/2107.04616/) (2021)
- [Estimating scale-invariant future in continuous time](https://mlsanity.com/paper/1802.06426/) (2018)

## Built on by (most cited first, 1 of 1)

- [Dynamical Priors as a Training Objective in Reinforcement Learning](https://mlsanity.com/paper/2604.21464/) (2026)

## Abstract (arXiv)

The ability to estimate temporal relationships is critical for both animals and artificial agents. Cognitive science and neuroscience provide remarkable insights into behavioral and neural aspects of temporal credit assignment. In particular, scale invariance of learning dynamics, observed in behavior and supported by neural data, is one of the key principles that governs animal perception: proportional rescaling of temporal relationships does not alter the overall learning efficiency. Here we integrate a computational neuroscience model of scale invariant memory into deep reinforcement learning (RL) agents. We first provide a theoretical analysis and then demonstrate through experiments that such agents can learn robustly across a wide range of temporal scales, unlike agents built with commonly used recurrent memory architectures such as LSTM. This result illustrates that incorporating computational principles from neuroscience and cognitive science into deep neural networks can enhance adaptability to complex temporal dynamics, mirroring some of the core properties of human learning.