# Learning to predict where to look in interactive environments using deep recurrent q-learning

arXiv: https://arxiv.org/abs/1612.05753 · HTML: https://mlsanity.com/paper/1612.05753/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2016.

## Built on by (most cited first, 4 of 4)

- [Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learning](https://mlsanity.com/paper/2209.09203/) (2022)
- [EyeFormer: Predicting Personalized Scanpaths with Transformer-Guided Reinforcement Learning](https://mlsanity.com/paper/2404.10163/) (2024)
- [Approximate information state based convergence analysis of recurrent Q-learning](https://mlsanity.com/paper/2306.05991/) (2023)
- [An Approach to Partial Observability in Games: Learning to Both Act and Observe](https://mlsanity.com/paper/2108.05701/) (2021)

## Abstract (arXiv)

Bottom-Up (BU) saliency models do not perform well in complex interactive environments where humans are actively engaged in tasks (e.g., sandwich making and playing the video games). In this paper, we leverage Reinforcement Learning (RL) to highlight task-relevant locations of input frames. We propose a soft attention mechanism combined with the Deep Q-Network (DQN) model to teach an RL agent how to play a game and where to look by focusing on the most pertinent parts of its visual input. Our evaluations on several Atari 2600 games show that the soft attention based model could predict fixation locations significantly better than bottom-up models such as Itti-Kochs saliency and Graph-Based Visual Saliency (GBVS) models.