mlsanityData to 14 Oct 2026

Transforming Game Play: A Comparative Study of DCQN and DTQN Architectures in Reinforcement Learning

arXiv 2410.10660 ↗First posted 14 Oct 2024cs.LG cs.AI cs.CVcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

In this study, we investigate the performance of Deep Q-Networks utilizing Convolutional Neural Networks (CNNs) and Transformer architectures across three different Atari games. The advent of DQNs has significantly advanced Reinforcement Learning, enabling agents to directly learn optimal policies from high-dimensional sensory inputs from pixel or RAM data. While CNN-based DQNs have been extensively studied and deployed in various domains, Transformer-based DQNs are relatively unexplored. Our research aims to fill this gap by benchmarking the performance of both DCQNs and DTQNs across the Atari games Asteroids, Space Invaders, and Centipede. We find that in the 35-40 million parameter range, the DCQN outperforms the DTQN in speed across both ViT and Projection Architectures. We also find the DCQN outperforms the DTQN in all games except for Centipede.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 13 references that are in the map. First posted on arXiv 14 Oct 2024.

Builds on

9 of 13 references.

References · 9

  1. 2017Attention Is All You Needcited by 60,361
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  5. 2015Prioritized Experience Replaycited by 1,099
  6. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  7. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
  8. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 269
  9. 2022Deep Transformer Q-Networks for Partially Observable Reinforcement Learningcited by 36

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024A Comparative Study of Deep Reinforcement Learning Models: DQN vs PPO vs A2C
  2. 2024FDQN: A Flexible Deep Q-Network Framework for Game Automation
  3. 2025Do We Need Transformers to Play FPS Video Games?
  4. 2024Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PC
  5. 2025Deep Reinforcement Learning in Applied Control: Challenges, Analysis, and Insights
  6. 2022Multi-Game Decision Transformers
  7. 2025Automatic Reward Shaping from Confounded Offline Data
  8. 2025Confounding Robust Deep Reinforcement Learning: A Causal Approach