# Visual Navigation with Spatial Attention

arXiv: https://arxiv.org/abs/2104.09807 · HTML: https://mlsanity.com/paper/2104.09807/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 38 references that are in the map.
- First posted on arXiv 20 Apr 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [Object Goal Navigation using Goal-Oriented Semantic Exploration](https://mlsanity.com/paper/2007.00643/) (2020)
- [Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05143/) (2016)
- [Stacked Attention Networks for Image Question Answering](https://mlsanity.com/paper/1511.02274/) (2015)
- [Image Captioning with Semantic Attention](https://mlsanity.com/paper/1603.03925/) (2016)
- [Cognitive Mapping and Planning for Visual Navigation](https://mlsanity.com/paper/1702.03920/) (2017)
- [Visual Semantic Navigation using Scene Priors](https://mlsanity.com/paper/1810.06543/) (2018)
- [Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning](https://mlsanity.com/paper/1812.00971/) (2018)
- [Visual Representations for Semantic Target Driven Navigation](https://mlsanity.com/paper/1805.06066/) (2018)
- [Learn To Pay Attention](https://mlsanity.com/paper/1804.02391/) (2018)
- [Gated-Attention Architectures for Task-Oriented Language Grounding](https://mlsanity.com/paper/1706.07230/) (2017)
- [Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering](https://mlsanity.com/paper/1511.05234/) (2015)
- [Learning Object Relation Graph and Tentative Policy for Visual Navigation](https://mlsanity.com/paper/2007.11018/) (2020)
- [Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Views](https://mlsanity.com/paper/2010.01191/) (2020)
- [Bayesian Relational Memory for Semantic Visual Navigation](https://mlsanity.com/paper/1909.04306/) (2019)
- [Factor Graph Attention](https://mlsanity.com/paper/1904.05880/) (2019)
- [Chasing Ghosts: Instruction Following as Bayesian State Tracking](https://mlsanity.com/paper/1907.02022/) (2019)
- [Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction](https://mlsanity.com/paper/1811.04179/) (2018)
- [Situational Fusion of Visual Representation for Visual Navigation](https://mlsanity.com/paper/1908.09073/) (2019)
- [Text-guided Attention Model for Image Captioning](https://mlsanity.com/paper/1612.03557/) (2016)
- [Perceptual Attention-based Predictive Control](https://mlsanity.com/paper/1904.11898/) (2019)
- [Simultaneous Mapping and Target Driven Navigation](https://mlsanity.com/paper/1911.07980/) (2019)

## Built on by (most cited first, 30 of 43)

- [PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning](https://mlsanity.com/paper/2201.10029/) (2022)
- [How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers](https://mlsanity.com/paper/2305.16925/) (2023)
- [One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestones](https://mlsanity.com/paper/2202.07028/) (2022)
- [NaviDiffusor: Cost-Guided Diffusion Model for Visual Navigation](https://mlsanity.com/paper/2504.10003/) (2025)
- [Object Memory Transformer for Object Goal Navigation](https://mlsanity.com/paper/2203.14708/) (2022)
- [Navigating to Objects in Unseen Environments by Distance Prediction](https://mlsanity.com/paper/2202.03735/) (2022)
- [Self-supervision through Random Segments with Autoregressive Coding (RandSAC)](https://mlsanity.com/paper/2203.12054/) (2022)
- [TopV-Nav: Unlocking the Top-View Spatial Reasoning Potential of MLLM for Zero-shot Object Navigation](https://mlsanity.com/paper/2411.16425/) (2024)
- [Towards Optimal Correlational Object Search](https://mlsanity.com/paper/2110.09991/) (2021)
- [REGNav: Room Expert Guided Image-Goal Navigation](https://mlsanity.com/paper/2502.10785/) (2025)
- [Personalized Instance-based Navigation Toward User-Specific Objects in Realistic Environments](https://mlsanity.com/paper/2410.18195/) (2024)
- [Multi-Agent Embodied Visual Semantic Navigation with Scene Prior Knowledge](https://mlsanity.com/paper/2109.09531/) (2021)
- [Video and Text Matching with Conditioned Embeddings](https://mlsanity.com/paper/2110.11298/) (2021)
- [Agent-Centric Relation Graph for Object Visual Navigation](https://mlsanity.com/paper/2111.14422/) (2021)
- [A System for Generalized 3D Multi-Object Search](https://mlsanity.com/paper/2303.03178/) (2023)
- [Aligning Knowledge Graph with Visual Perception for Object-goal Navigation](https://mlsanity.com/paper/2402.18892/) (2024)
- [What do navigation agents learn about their environment?](https://mlsanity.com/paper/2206.08500/) (2022)
- [A Visual Active Search Framework for Geospatial Exploration](https://mlsanity.com/paper/2211.15788/) (2022)
- [A Partially Supervised Reinforcement Learning Framework for Visual Active Search](https://mlsanity.com/paper/2310.09689/) (2023)
- [TDANet: Target-Directed Attention Network For Object-Goal Visual Navigation With Zero-Shot Ability](https://mlsanity.com/paper/2404.08353/) (2024)
- [Recent Advancements in Deep Learning Applications and Methods for Autonomous Navigation: A Comprehensive Review](https://mlsanity.com/paper/2302.11089/) (2023)
- [Spot the Difference: A Novel Task for Embodied Agents in Changing Environments](https://mlsanity.com/paper/2204.08502/) (2022)
- [Implicit Obstacle Map-driven Indoor Navigation Model for Robust Obstacle Avoidance](https://mlsanity.com/paper/2308.12845/) (2023)
- [NOLO: Navigate Only Look Once](https://mlsanity.com/paper/2408.01384/) (2024)
- [Flex: End-to-End Text-Instructed Visual Navigation from Foundation Model Features](https://mlsanity.com/paper/2410.13002/) (2024)
- [Representation Learning for Continuous Action Spaces is Beneficial for Efficient Policy Learning](https://mlsanity.com/paper/2211.13257/) (2022)
- [Building Category Graphs Representation with Spatial and Temporal Attention for Visual Navigation](https://mlsanity.com/paper/2312.03327/) (2023)
- [STRNet: Visual Navigation with Spatio-Temporal Representation through Dynamic Graph Aggregation](https://mlsanity.com/paper/2604.02829/) (2026)
- [Symmetry-aware Neural Architecture for Embodied Visual Navigation](https://mlsanity.com/paper/2112.09515/) (2021)
- [Unsupervised Visual Odometry and Action Integration for PointGoal Navigation in Indoor Environment](https://mlsanity.com/paper/2210.00413/) (2022)

## Abstract (arXiv)

This work focuses on object goal visual navigation, aiming at finding the location of an object from a given class, where in each step the agent is provided with an egocentric RGB image of the scene. We propose to learn the agent's policy using a reinforcement learning algorithm. Our key contribution is a novel attention probability model for visual navigation tasks. This attention encodes semantic information about observed objects, as well as spatial information about their place. This combination of the "what" and the "where" allows the agent to navigate toward the sought-after object effectively. The attention model is shown to improve the agent's policy and to achieve state-of-the-art results on commonly-used datasets.