# Towards self-attention based visual navigation in the real world

arXiv: https://arxiv.org/abs/2209.07043 · HTML: https://mlsanity.com/paper/2209.07043/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 15 Sep 2022.

## Builds on (most cited first, 18 of 18)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping](https://mlsanity.com/paper/1709.07857/) (2017)
- [Towards Interpretable Reinforcement Learning Using Attention Augmented Agents](https://mlsanity.com/paper/1906.02500/) (2019)
- [Neuroevolution of Self-Interpretable Agents](https://mlsanity.com/paper/2003.08165/) (2020)
- [VR-Goggles for Robots: Real-to-sim Domain Adaptation for Visual Control](https://mlsanity.com/paper/1802.00265/) (2018)
- [SplitNet: Sim2Sim and Task2Task Transfer for Embodied Visual Navigation](https://mlsanity.com/paper/1905.07512/) (2019)
- [A Data-Efficient Framework for Training and Sim-to-Real Transfer of Navigation Policies](https://mlsanity.com/paper/1810.04871/) (2018)
- [Teaching a Machine to Read Maps with Deep Reinforcement Learning](https://mlsanity.com/paper/1711.07479/) (2017)
- [GeneSIS-RT: Generating Synthetic Images for training Secondary Real-world Tasks](https://mlsanity.com/paper/1710.04280/) (2017)
- [Using RGB Image as Visual Input for Mapless Robot Navigation](https://mlsanity.com/paper/1903.09927/) (2019)
- [Generative NeuroEvolution for Deep Learning](https://mlsanity.com/paper/1312.5355/) (2013)
- [Attend Before you Act: Leveraging human visual attention for continual learning](https://mlsanity.com/paper/1807.09664/) (2018)

## Built on by (most cited first, 1 of 1)

- [Spatially-Enhanced Recurrent Memory for Long-Range Mapless Navigation via End-to-End Reinforcement Learning](https://mlsanity.com/paper/2506.05997/) (2025)

## Abstract (arXiv)

Vision guided navigation requires processing complex visual information to inform task-orientated decisions. Applications include autonomous robots, self-driving cars, and assistive vision for humans. A key element is the extraction and selection of relevant features in pixel space upon which to base action choices, for which Machine Learning techniques are well suited. However, Deep Reinforcement Learning agents trained in simulation often exhibit unsatisfactory results when deployed in the real-world due to perceptual differences known as the textitreality gap. An approach that is yet to be explored to bridge this gap is self-attention. In this paper we (1) perform a systematic exploration of the hyperparameter space for self-attention based navigation of 3D environments and qualitatively appraise behaviour observed from different hyperparameter sets, including their ability to generalise; (2) present strategies to improve the agents' generalisation abilities and navigation behaviour; and (3) show how models trained in simulation are capable of processing real world images meaningfully in real time. To our knowledge, this is the first demonstration of a self-attention based agent successfully trained in navigating a 3D action space, using less than 4000 parameters.