# Estimating scale-invariant future in continuous time

arXiv: https://arxiv.org/abs/1802.06426 · HTML: https://mlsanity.com/paper/1802.06426/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2018.

## Built on by (most cited first, 4 of 4)

- [Deep reinforcement learning with time-scale invariant memory](https://mlsanity.com/paper/2412.15292/) (2024)
- [Gradual Forgetting: Logarithmic Compression for Extending Transformer Context Windows](https://mlsanity.com/paper/2510.22109/) (2025)
- [Predicting the future with a scale-invariant temporal memory for the past](https://mlsanity.com/paper/2101.10953/) (2021)
- [Graceful forgetting: Memory as a process](https://mlsanity.com/paper/2502.11105/) (2025)

## Abstract (arXiv)

Natural learners must compute an estimate of future outcomes that follow from a stimulus in continuous time. Widely used reinforcement learning algorithms discretize continuous time and estimate either transition functions from one step to the next (model-based algorithms) or a scalar value of exponentially-discounted future reward using the Bellman equation (model-free algorithms). An important drawback of model-based algorithms is that computational cost grows linearly with the amount of time to be simulated. On the other hand, an important drawback of model-free algorithms is the need to select a time-scale required for exponential discounting. We present a computational mechanism, developed based on work in psychology and neuroscience, for computing a scale-invariant timeline of future outcomes. This mechanism efficiently computes an estimate of inputs as a function of future time on a logarithmically-compressed scale, and can be used to generate a scale-invariant power-law-discounted estimate of expected future reward. The representation of future time retains information about what will happen when. The entire timeline can be constructed in a single parallel operation which generates concrete behavioral and neural predictions. This computational mechanism could be incorporated into future reinforcement learning algorithms.