# Deep Reinforcement Learning based Automatic Exploration for Navigation in Unknown Environment

arXiv: https://arxiv.org/abs/2007.11808 · HTML: https://mlsanity.com/paper/2007.11808/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2020.

## Built on by (most cited first, 30 of 34)

- [A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directions](https://mlsanity.com/paper/2406.05785/) (2024)
- [TrajGen: Generating Realistic and Diverse Trajectories with Reactive and Feasible Agent Behaviors for Autonomous Driving](https://mlsanity.com/paper/2203.16792/) (2022)
- [Deep Reinforcement Learning-based Large-scale Robot Exploration](https://mlsanity.com/paper/2403.10833/) (2024)
- [Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation](https://mlsanity.com/paper/2109.00525/) (2021)
- [Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualization](https://mlsanity.com/paper/2209.00347/) (2022)
- [Deep Learning for Visual Localization and Mapping: A Survey](https://mlsanity.com/paper/2308.14039/) (2023)
- [DARE: Diffusion Policy for Autonomous Robot Exploration](https://mlsanity.com/paper/2410.16687/) (2024)
- [Generalizing Consistency Policy to Visual RL with Prioritized Proximal Experience Regularization](https://mlsanity.com/paper/2410.00051/) (2024)
- [Dream to Drive with Predictive Individual World Model](https://mlsanity.com/paper/2501.16733/) (2025)
- [Multi-task Safe Reinforcement Learning for Navigating Intersections in Dense Traffic](https://mlsanity.com/paper/2202.09644/) (2022)
- [TeViR: Text-to-Video Reward with Diffusion Models for Efficient Reinforcement Learning](https://mlsanity.com/paper/2505.19769/) (2025)
- [Deep-Attack over the Deep Reinforcement Learning](https://mlsanity.com/paper/2205.00807/) (2022)
- [HDPlanner: Advancing Autonomous Deployments in Unknown Environments through Hierarchical Decision Networks](https://mlsanity.com/paper/2408.03768/) (2024)
- [Deep Reinforcement Learning-based UAV Navigation and Control: A Soft Actor-Critic with Hindsight Experience Replay Approach](https://mlsanity.com/paper/2106.01016/) (2021)
- [Sample-efficient Unsupervised Policy Cloning from Ensemble Self-supervised Labeled Videos](https://mlsanity.com/paper/2412.10778/) (2024)
- [Conditional Goal-oriented Trajectory Prediction for Interacting Vehicles with Vectorized Representation](https://mlsanity.com/paper/2210.15449/) (2022)
- [Deep Reinforcement Learning with Explicit Context Representation](https://mlsanity.com/paper/2310.09924/) (2023)
- [Learning Future Representation with Synthetic Observations for Sample-efficient Reinforcement Learning](https://mlsanity.com/paper/2405.11740/) (2024)
- [A Survey on Reinforcement Learning Applications in SLAM](https://mlsanity.com/paper/2408.14518/) (2024)
- [OPT-GAN: A Broad-Spectrum Global Optimizer for Black-box Problems by Learning Distribution](https://mlsanity.com/paper/2102.03888/) (2021)
- [Efficient Bayesian Policy Reuse with a Scalable Observation Model in Deep Reinforcement Learning](https://mlsanity.com/paper/2204.07729/) (2022)
- [Salience-Invariant Consistent Policy Learning for Generalization in Visual Reinforcement Learning](https://mlsanity.com/paper/2502.08336/) (2025)
- [MapExRL: Human-Inspired Indoor Exploration with Predicted Environment Context and Reinforcement Learning](https://mlsanity.com/paper/2503.01548/) (2025)
- [Depthwise Convolution for Multi-Agent Communication with Enhanced Mean-Field Approximation](https://mlsanity.com/paper/2203.02896/) (2022)
- [Simultaneous Double Q-learning with Conservative Advantage Learning for Actor-Critic Methods](https://mlsanity.com/paper/2205.03819/) (2022)
- [A Fully Controllable Agent in the Path Planning using Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2205.09967/) (2022)
- [Task-Agnostic Learning to Accomplish New Tasks](https://mlsanity.com/paper/2209.04100/) (2022)
- [MSVIPER: Improved Policy Distillation for Reinforcement-Learning-Based Robot Navigation](https://mlsanity.com/paper/2209.09079/) (2022)
- [Practical Probabilistic Model-based Deep Reinforcement Learning by Integrating Dropout Uncertainty and Trajectory Sampling](https://mlsanity.com/paper/2309.11089/) (2023)
- [ComSD: Balancing Behavioral Quality and Diversity in Unsupervised Skill Discovery](https://mlsanity.com/paper/2309.17203/) (2023)

## Abstract (arXiv)

This paper investigates the automatic exploration problem under the unknown environment, which is the key point of applying the robotic system to some social tasks. The solution to this problem via stacking decision rules is impossible to cover various environments and sensor properties. Learning based control methods are adaptive for these scenarios. However, these methods are damaged by low learning efficiency and awkward transferability from simulation to reality. In this paper, we construct a general exploration framework via decomposing the exploration process into the decision, planning, and mapping modules, which increases the modularity of the robotic system. Based on this framework, we propose a deep reinforcement learning based decision algorithm which uses a deep neural network to learning exploration strategy from the partial map. The results show that this proposed algorithm has better learning efficiency and adaptability for unknown environments. In addition, we conduct the experiments on the physical robot, and the results suggest that the learned policy can be well transfered from simulation to the real robot.