# Gamma-Nets: Generalizing Value Estimation over Timescale

arXiv: https://arxiv.org/abs/1911.07794 · HTML: https://mlsanity.com/paper/1911.07794/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2019.

## Built on by (most cited first, 6 of 6)

- [Heuristic-Guided Reinforcement Learning](https://mlsanity.com/paper/2106.02757/) (2021)
- [On the Role of Discount Factor in Offline Reinforcement Learning](https://mlsanity.com/paper/2206.03383/) (2022)
- [What Should I Know? Using Meta-gradient Descent for Predictive Feature Discovery in a Single Stream of Experience](https://mlsanity.com/paper/2206.06485/) (2022)
- [On the Value of Myopic Behavior in Policy Reuse](https://mlsanity.com/paper/2305.17623/) (2023)
- [Hierarchical Reinforcement Learning Framework for Adaptive Walking Control Using General Value Functions of Lower-Limb Sensor Signals](https://mlsanity.com/paper/2507.16983/) (2025)
- [Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2610.00849/) (2026)

## Abstract (arXiv)

We present Gamma-nets, a method for generalizing value function estimation over timescale. By using the timescale as one of the estimator's inputs we can estimate value for arbitrary timescales. As a result, the prediction target for any timescale is available and we are free to train on multiple timescales at each timestep. Here we empirically evaluate Gamma-nets in the policy evaluation setting. We first demonstrate the approach on a square wave and then on a robot arm using linear function approximation. Next, we consider the deep reinforcement learning setting using several Atari video games. Our results show that Gamma-nets can be effective for predicting arbitrary timescales, with only a small cost in accuracy as compared to learning estimators for fixed timescales. Gamma-nets provide a method for compactly making predictions at many timescales without requiring a priori knowledge of the task, making it a valuable contribution to ongoing work on model-based planning, representation learning, and lifelong learning algorithms.