# Interval timing in deep reinforcement learning agents

arXiv: https://arxiv.org/abs/1905.13469 · HTML: https://mlsanity.com/paper/1905.13469/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2019.

## Built on by (most cited first, 7 of 7)

- [Circumventing interpretability: How to defeat mind-readers](https://mlsanity.com/paper/2212.11415/) (2022)
- [Unraveling the Complexity of Memory in RL Agents: an Approach for Classification and Evaluation](https://mlsanity.com/paper/2412.06531/) (2024)
- [Bayesian sense of time in biological and artificial brains](https://mlsanity.com/paper/2201.05464/) (2022)
- [Deep reinforcement learning with time-scale invariant memory](https://mlsanity.com/paper/2412.15292/) (2024)
- [Latent Time-Adaptive Drift-Diffusion Model](https://mlsanity.com/paper/2106.02742/) (2021)
- [Emergent time-keeping mechanisms in a deep reinforcement learning agent performing an interval timing task](https://mlsanity.com/paper/2508.15784/) (2025)
- [Modulation of temporal decision-making in a deep reinforcement learning agent under the dual-task paradigm](https://mlsanity.com/paper/2511.01415/) (2025)

## Abstract (arXiv)

The measurement of time is central to intelligent behavior. We know that both animals and artificial agents can successfully use temporal dependencies to select actions. In artificial agents, little work has directly addressed (1) which architectural components are necessary for successful development of this ability, (2) how this timing ability comes to be represented in the units and actions of the agent, and (3) whether the resulting behavior of the system converges on solutions similar to those of biology. Here we studied interval timing abilities in deep reinforcement learning agents trained end-to-end on an interval reproduction paradigm inspired by experimental literature on mechanisms of timing. We characterize the strategies developed by recurrent and feedforward agents, which both succeed at temporal reproduction using distinct mechanisms, some of which bear specific and intriguing similarities to biological systems. These findings advance our understanding of how agents come to represent time, and they highlight the value of experimentally inspired approaches to characterizing agent abilities.