# Terminal Prediction as an Auxiliary Task for Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1907.10827 · HTML: https://mlsanity.com/paper/1907.10827/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2019.

## Built on by (most cited first, 11 of 11)

- [On The Effect of Auxiliary Tasks on Representation Dynamics](https://mlsanity.com/paper/2102.13089/) (2021)
- [A Survey of State Representation Learning for Deep Reinforcement Learning](https://mlsanity.com/paper/2506.17518/) (2025)
- [Continual Auxiliary Task Learning](https://mlsanity.com/paper/2202.11133/) (2022)
- [Learning State Representations from Random Deep Action-conditional Predictions](https://mlsanity.com/paper/2102.04897/) (2021)
- [Explainability Via Causal Self-Talk](https://mlsanity.com/paper/2211.09937/) (2022)
- [The Curse of Diversity in Ensemble-Based Exploration](https://mlsanity.com/paper/2405.04342/) (2024)
- [Emergent Risk Awareness in Rational Agents under Resource Constraints](https://mlsanity.com/paper/2505.23436/) (2025)
- [Investigating Navigation Strategies in the Morris Water Maze through Deep Reinforcement Learning](https://mlsanity.com/paper/2306.01066/) (2023)
- [Ensemble and Auxiliary Tasks for Data-Efficient Deep Reinforcement Learning](https://mlsanity.com/paper/2107.01904/) (2021)
- [Learning to Terminate in Object Navigation](https://mlsanity.com/paper/2309.16164/) (2023)
- [An Advanced Reinforcement Learning Framework for Online Scheduling of Deferrable Workloads in Cloud Computing](https://mlsanity.com/paper/2406.01047/) (2024)

## Abstract (arXiv)

Deep reinforcement learning has achieved great successes in recent years, but there are still open challenges, such as convergence to locally optimal policies and sample inefficiency. In this paper, we contribute a novel self-supervised auxiliary task, i.e., Terminal Prediction (TP), estimating temporal closeness to terminal states for episodic tasks. The intuition is to help representation learning by letting the agent predict how close it is to a terminal state, while learning its control policy. Although TP could be integrated with multiple algorithms, this paper focuses on Asynchronous Advantage Actor-Critic (A3C) and demonstrating the advantages of A3C-TP. Our extensive evaluation includes: a set of Atari games, the BipedalWalker domain, and a mini version of the recently proposed multi-agent Pommerman game. Our results on Atari games and the BipedalWalker domain suggest that A3C-TP outperforms standard A3C in most of the tested domains and in others it has similar performance. In Pommerman, our proposed method provides significant improvement both in learning efficiency and converging to better policies against different opponents.