# Using RGB Image as Visual Input for Mapless Robot Navigation

arXiv: https://arxiv.org/abs/1903.09927 · HTML: https://mlsanity.com/paper/1903.09927/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2019.

## Built on by (most cited first, 5 of 5)

- [HabiCrowd: A High Performance Simulator for Crowd-Aware Visual Navigation](https://mlsanity.com/paper/2306.11377/) (2023)
- [Towards self-attention based visual navigation in the real world](https://mlsanity.com/paper/2209.07043/) (2022)
- [Robot path planning using deep reinforcement learning](https://mlsanity.com/paper/2302.09120/) (2023)
- [GoToNet: Fast Monocular Scene Exposure and Exploration](https://mlsanity.com/paper/2206.05967/) (2022)
- [An experimental evaluation of Siamese Neural Networks for robot localization using omnidirectional imaging in indoor environments](https://mlsanity.com/paper/2407.10536/) (2024)

## Abstract (arXiv)

Robot navigation in mapless environment is one of the essential problems and challenges in mobile robots. Deep reinforcement learning is a promising technique to tackle the task of mapless navigation. Since reinforcement learning requires a lot of explorations, it is usually necessary to train the agent in the simulator and then migrate to the real environment. The big reality gap makes RGB image, the most common visual sensor, rarely used. In this paper we present a learning-based mapless motion planner by taking RGB images as visual inputs. Many parameters in end-to-end navigation network taking RGB images as visual input are used to extract visual features. Therefore, we decouple visual features extracted module from the reinforcement learning network to reduce the need of interactions between agent and environment. We use Variational Autoencoder (VAE) to encode the image, and input the obtained latent vector as low-dimensional visual features into the network together with the target and motion information, so that the sampling efficiency of the agent is greatly improved. We built simulation environment as robot navigation environment for algorithm comparison. In the test environment, the proposed method was compared with the end-to-end network, which proved its effectiveness and efficiency. The source code is available: https://github.com/marooncn/navbot.