mlsanityData to 14 Oct 2026

Do We Need Transformers to Play FPS Video Games?

arXiv 2504.17891 ↗First posted 24 Apr 2025cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

also in 1 more
Abstract (arXiv)

In this paper, we explore the Transformer based architectures for reinforcement learning in both online and offline settings within the Doom game environment. Our investigation focuses on two primary approaches: Deep Transformer Q- learning Networks (DTQN) for online learning and Decision Transformers (DT) for offline reinforcement learning. DTQN leverages the sequential modelling capabilities of Transformers to enhance Q-learning in partially observable environments,while Decision Transformers repurpose sequence modelling techniques to enable offline agents to learn from past trajectories without direct interaction with the environment. We conclude that while Transformers might have performed well in Atari games, more traditional methods perform better than Transformer based method in both the settings in the VizDoom environment.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 12 of its 14 references that are in the map. First posted on arXiv 24 Apr 2025.

Builds on

12 of 14 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Decision Mamba: Reinforcement Learning via Sequence Modeling with Selective State Spaces29 Mar
  2. 2023Recurrent Action Transformer with Memory15 Jun
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 9

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  5. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 231
  6. 2016ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learningcited by 179
  7. 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
  8. 2022Deep Transformer Q-Networks for Partially Observable Reinforcement Learningcited by 36
  9. 2019Deep Reinforcement Learning on a Budget: 3D Control and Reasoning Without a Supercomputercited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Transforming Game Play: A Comparative Study of DCQN and DTQN Architectures in Reinforcement Learning
  2. 2022Multi-Game Decision Transformers
  3. 2025A Comparison Between Decision Transformers and Traditional Offline Reinforcement Learning Algorithms
  4. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions
  5. 2025A New Perspective on Transformers in Online Reinforcement Learning for Continuous Control
  6. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL
  7. 2026Q-learning Penalized Transformer for Safe Offline Reinforcement Learning
  8. 2024Q-value Regularized Transformer for Offline Reinforcement Learning