# Attention-based Curiosity-driven Exploration in Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1910.10840 · HTML: https://mlsanity.com/paper/1910.10840/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2019.

## Built on by (most cited first, 5 of 5)

- [From Curiosity to Caution: Mitigating Reward Hacking for Best-of-N with Pessimism](https://mlsanity.com/paper/2604.04648/) (2026)
- [Policy Augmentation: An Exploration Strategy for Faster Convergence of Deep Reinforcement Learning Algorithms](https://mlsanity.com/paper/2102.05249/) (2021)
- [Ensemble Reinforcement Learning in Continuous Spaces -- A Hierarchical Multi-Step Approach for Policy Training](https://mlsanity.com/paper/2209.14488/) (2022)
- [Learning Rhetorical Structure Theory-based descriptions of observed behaviour](https://mlsanity.com/paper/2206.12294/) (2022)
- [Replication of Multi-agent Reinforcement Learning for the "Hide and Seek" Problem](https://mlsanity.com/paper/2310.05430/) (2023)

## Abstract (arXiv)

Reinforcement Learning enables to train an agent via interaction with the environment. However, in the majority of real-world scenarios, the extrinsic feedback is sparse or not sufficient, thus intrinsic reward formulations are needed to successfully train the agent. This work investigates and extends the paradigm of curiosity-driven exploration. First, a probabilistic approach is taken to exploit the advantages of the attention mechanism, which is successfully applied in other domains of Deep Learning. Combining them, we propose new methods, such as AttA2C, an extension of the Actor-Critic framework. Second, another curiosity-based approach - ICM - is extended. The proposed model utilizes attention to emphasize features for the dynamic models within ICM, moreover, we also modify the loss function, resulting in a new curiosity formulation, which we call rational curiosity. The corresponding implementation can be found at https://github.com/rpatrik96/AttA2C/.