# Temporal Shift Reinforcement Learning

arXiv: https://arxiv.org/abs/2109.02145 · HTML: https://mlsanity.com/paper/2109.02145/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 5 Sep 2021.

## Builds on (most cited first, 16 of 16)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://mlsanity.com/paper/2004.04136/) (2020)
- [Reinforcement Learning with Augmented Data](https://mlsanity.com/paper/2004.14990/) (2020)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [ECO: Efficient Convolutional Network for Online Video Understanding](https://mlsanity.com/paper/1804.09066/) (2018)
- [Reinforcement Learning with Latent Flow](https://mlsanity.com/paper/2101.01857/) (2021)
- [Path Integral Guided Policy Search](https://mlsanity.com/paper/1610.00529/) (2016)
- [Motion Perception in Reinforcement Learning with Dynamic Objects](https://mlsanity.com/paper/1901.03162/) (2019)
- [A Vision Based Deep Reinforcement Learning Algorithm for UAV Obstacle Avoidance](https://mlsanity.com/paper/2103.06403/) (2021)
- [Efficient Object Detection Model for Real-Time UAV Applications](https://mlsanity.com/paper/1906.00786/) (2019)

## Abstract (arXiv)

The function approximators employed by traditional image-based Deep Reinforcement Learning (DRL) algorithms usually lack a temporal learning component and instead focus on learning the spatial component. We propose a technique, Temporal Shift Reinforcement Learning (TSRL), wherein both temporal, as well as spatial components are jointly learned. Moreover, TSRL does not require additional parameters to perform temporal learning. We show that TSRL outperforms the commonly used frame stacking heuristic on both of the Atari environments we test on while beating the SOTA for one of them. This investigation has implications in the robotics as well as sequential decision-making domains.