# Towards Cognitive Exploration through Deep Reinforcement Learning for Mobile Robots

arXiv: https://arxiv.org/abs/1610.01733 · HTML: https://mlsanity.com/paper/1610.01733/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2016.

## Built on by (most cited first, 11 of 11)

- [Depth-CUPRL: Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles](https://mlsanity.com/paper/2206.15211/) (2022)
- [Deep Reinforcement Learning with Enhanced PPO for Safe Mobile Robot Navigation](https://mlsanity.com/paper/2405.16266/) (2024)
- [Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular Decomposition](https://mlsanity.com/paper/2110.09018/) (2021)
- [TopoNav: Topological Navigation for Efficient Exploration in Sparse Reward Environments](https://mlsanity.com/paper/2402.04061/) (2024)
- [Deep Reinforcement Learning for Mobile Robot Path Planning](https://mlsanity.com/paper/2404.06974/) (2024)
- [Vision-Based Mobile Robotics Obstacle Avoidance With Deep Reinforcement Learning](https://mlsanity.com/paper/2103.04727/) (2021)
- [Deep Learning for Embodied Vision Navigation: A Survey](https://mlsanity.com/paper/2108.04097/) (2021)
- [Double Deep Reinforcement Learning Techniques for Low Dimensional Sensing Mapless Navigation of Terrestrial Mobile Robots](https://mlsanity.com/paper/2301.11173/) (2023)
- [Enhanced Low-Dimensional Sensing Mapless Navigation of Terrestrial Mobile Robots Using Double Deep Reinforcement Learning Techniques](https://mlsanity.com/paper/2310.13809/) (2023)
- [Parallel Distributional Deep Reinforcement Learning for Mapless Navigation of Terrestrial Mobile Robots](https://mlsanity.com/paper/2408.05744/) (2024)
- [BAMAX: Backtrack Assisted Multi-Agent Exploration using Reinforcement Learning](https://mlsanity.com/paper/2411.08400/) (2024)

## Abstract (arXiv)

Exploration in an unknown environment is the core functionality for mobile robots. Learning-based exploration methods, including convolutional neural networks, provide excellent strategies without human-designed logic for the feature extraction. But the conventional supervised learning algorithms cost lots of efforts on the labeling work of datasets inevitably. Scenes not included in the training set are mostly unrecognized either. We propose a deep reinforcement learning method for the exploration of mobile robots in an indoor environment with the depth information from an RGB-D sensor only. Based on the Deep Q-Network framework, the raw depth image is taken as the only input to estimate the Q values corresponding to all moving commands. The training of the network weights is end-to-end. In arbitrarily constructed simulation environments, we show that the robot can be quickly adapted to unfamiliar scenes without any man-made labeling. Besides, through analysis of receptive fields of feature representations, deep reinforcement learning motivates the convolutional networks to estimate the traversability of the scenes. The test results are compared with the exploration strategies separately based on deep learning or reinforcement learning. Even trained only in the simulated environment, experimental results in real-world environment demonstrate that the cognitive ability of robot controller is dramatically improved compared with the supervised method. We believe it is the first time that raw sensor information is used to build cognitive exploration strategy for mobile robots through end-to-end deep reinforcement learning.