# Transformer Based Reinforcement Learning For Games

arXiv: https://arxiv.org/abs/1912.03918 · HTML: https://mlsanity.com/paper/1912.03918/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2019.

## Built on by (most cited first, 5 of 5)

- [Deep Transformer Q-Networks for Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2206.01078/) (2022)
- [Optimal Return-to-Go Guided Decision Transformer for Auto-Bidding in Advertisement](https://mlsanity.com/paper/2506.21956/) (2025)
- [Hierarchical RNNs-Based Transformers MADDPG for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/2105.04888/) (2021)
- [PCGPT: Procedural Content Generation via Transformers](https://mlsanity.com/paper/2310.02405/) (2023)
- [Smart Sampling: Self-Attention and Bootstrapping for Improved Ensembled Q-Learning](https://mlsanity.com/paper/2405.08252/) (2024)

## Abstract (arXiv)

Recent times have witnessed sharp improvements in reinforcement learning tasks using deep reinforcement learning techniques like Deep Q Networks, Policy Gradients, Actor Critic methods which are based on deep learning based models and back-propagation of gradients to train such models. An active area of research in reinforcement learning is about training agents to play complex video games, which so far has been something accomplished only by human intelligence. Some state of the art performances in video game playing using deep reinforcement learning are obtained by processing the sequence of frames from video games, passing them through a convolutional network to obtain features and then using recurrent neural networks to figure out the action leading to optimal rewards. The recurrent neural network will learn to extract the meaningful signal out of the sequence of such features. In this work, we propose a method utilizing a transformer network which have recently replaced RNNs in Natural Language Processing (NLP), and perform experiments to compare with existing methods.