# Deep Attention Recurrent Q-Network

arXiv: https://arxiv.org/abs/1512.01693 · HTML: https://mlsanity.com/paper/1512.01693/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2015.

## Built on by (most cited first, 27 of 27)

- [Deep Transformer Q-Networks for Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2206.01078/) (2022)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Sample-efficient Reinforcement Learning Representation Learning with Curiosity Contrastive Forward Dynamics Model](https://mlsanity.com/paper/2103.08255/) (2021)
- [FRAMU: Attention-based Machine Unlearning using Federated Reinforcement Learning](https://mlsanity.com/paper/2309.10283/) (2023)
- [Visual Explanation using Attention Mechanism in Actor-Critic-based Deep Reinforcement Learning](https://mlsanity.com/paper/2103.04067/) (2021)
- [Action Candidate Driven Clipped Double Q-learning for Discrete and Continuous Action Tasks](https://mlsanity.com/paper/2203.11526/) (2022)
- [Attention Option-Critic](https://mlsanity.com/paper/2201.02628/) (2022)
- [Temporal-Spatial Causal Interpretations for Vision-Based Reinforcement Learning](https://mlsanity.com/paper/2112.03020/) (2021)
- [Transformer in Transformer as Backbone for Deep Reinforcement Learning](https://mlsanity.com/paper/2212.14538/) (2022)
- [On learning history based policies for controlling Markov decision processes](https://mlsanity.com/paper/2211.03011/) (2022)
- [Approximate information state based convergence analysis of recurrent Q-learning](https://mlsanity.com/paper/2306.05991/) (2023)
- [Unraveling the Complexity of Memory in RL Agents: an Approach for Classification and Evaluation](https://mlsanity.com/paper/2412.06531/) (2024)
- [Improving Sample Efficiency of Value Based Models Using Attention and Vision Transformers](https://mlsanity.com/paper/2202.00710/) (2022)
- [Emerging Threats in Deep Learning-Based Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2210.11237/) (2022)
- [Don't Watch Me: A Spatio-Temporal Trojan Attack on Deep-Reinforcement-Learning-Augment Autonomous Driving](https://mlsanity.com/paper/2211.14440/) (2022)
- [Pretrained Encoders are All You Need](https://mlsanity.com/paper/2106.05139/) (2021)
- [Deep Reinforcement Learning with Swin Transformers](https://mlsanity.com/paper/2206.15269/) (2022)
- [Automatic Deduction Path Learning via Reinforcement Learning with Environmental Correction](https://mlsanity.com/paper/2306.10083/) (2023)
- [Socially aware navigation for mobile robots: a survey on deep reinforcement learning approaches](https://mlsanity.com/paper/2512.00049/) (2025)
- [Playing Atari Space Invaders with Sparse Cosine Optimized Policy Evolution](https://mlsanity.com/paper/2508.08526/) (2025)
- [Neural Networks and Denotation](https://mlsanity.com/paper/2103.08315/) (2021)
- [Active Vision Reinforcement Learning under Limited Visual Observability](https://mlsanity.com/paper/2306.00975/) (2023)
- [RigLSTM: Recurrent Independent Grid LSTM for Generalizable Sequence Learning](https://mlsanity.com/paper/2311.02123/) (2023)
- [Portfolio Management using Deep Reinforcement Learning](https://mlsanity.com/paper/2405.01604/) (2024)
- [Pay Attention to What and Where? Interpretable Feature Extractor in Vision-based Deep Reinforcement Learning](https://mlsanity.com/paper/2504.10071/) (2025)
- [Bi-directional Recurrence Improves Transformer in Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2505.11153/) (2025)
- [Optimizing Sensory Neurons: Nonlinear Attention Mechanisms for Accelerated Convergence in Permutation-Invariant Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/2506.00691/) (2025)

## Abstract (arXiv)

A deep learning approach to reinforcement learning led to a general learner able to train on visual input to play a variety of arcade games at the human and superhuman levels. Its creators at the Google DeepMind's team called the approach: Deep Q-Network (DQN). We present an extension of DQN by "soft" and "hard" attention mechanisms. Tests of the proposed Deep Attention Recurrent Q-Network (DARQN) algorithm on multiple Atari 2600 games show level of performance superior to that of DQN. Moreover, built-in attention mechanisms allow a direct online monitoring of the training process by highlighting the regions of the game screen the agent is focusing on when making decisions.