# Passing Through Narrow Gaps with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2103.03991 · HTML: https://mlsanity.com/paper/2103.03991/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 6 Mar 2021.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Reinforcement Learning with Unsupervised Auxiliary Tasks](https://mlsanity.com/paper/1611.05397/) (2016)
- [Cognitive Mapping and Planning for Visual Navigation](https://mlsanity.com/paper/1702.03920/) (2017)
- [BADGR: An Autonomous Self-Supervised Learning-Based Navigation System](https://mlsanity.com/paper/2002.05700/) (2020)
- [Combining Optimal Control and Learning for Visual Navigation in Novel Environments](https://mlsanity.com/paper/1903.02531/) (2019)
- [Guided Curriculum Learning for Walking Over Complex Terrain](https://mlsanity.com/paper/2010.03848/) (2020)
- [A Comparative Study of Bug Algorithms for Robot Navigation](https://mlsanity.com/paper/1808.05050/) (2018)
- [Virtual Surfaces and Attitude Aware Planning and Behaviours for Negative Obstacle Navigation](https://mlsanity.com/paper/2010.16018/) (2020)
- [Context Aware Robot Navigation using Interactively Built Semantic Maps](https://mlsanity.com/paper/1710.08682/) (2017)
- [Learning When to Switch: Composing Controllers to Traverse a Sequence of Terrain Artifacts](https://mlsanity.com/paper/2011.00440/) (2020)
- [Learning Setup Policies: Reliable Transition Between Locomotion Behaviours](https://mlsanity.com/paper/2101.09391/) (2021)
- [Multiplicative Controller Fusion: Leveraging Algorithmic Priors for Sample-efficient Reinforcement Learning and Safe Sim-To-Real Transfer](https://mlsanity.com/paper/2003.05117/) (2020)

## Built on by (most cited first, 1 of 1)

- [Mixing Human Demonstrations with Self-Exploration in Experience Replay for Deep Reinforcement Learning](https://mlsanity.com/paper/2107.06840/) (2021)

## Abstract (arXiv)

The U.S. Defense Advanced Research Projects Agency (DARPA) Subterranean Challenge requires teams of robots to traverse difficult and diverse underground environments. Traversing small gaps is one of the challenging scenarios that robots encounter. Imperfect sensor information makes it difficult for classical navigation methods, where behaviours require significant manual fine tuning. In this paper we present a deep reinforcement learning method for autonomously navigating through small gaps, where contact between the robot and the gap may be required. We first learn a gap behaviour policy to get through small gaps (only centimeters wider than the robot). We then learn a goal-conditioned behaviour selection policy that determines when to activate the gap behaviour policy. We train our policies in simulation and demonstrate their effectiveness with a large tracked robot in simulation and on the real platform. In simulation experiments, our approach achieves 93% success rate when the gap behaviour is activated manually by an operator, and 63% with autonomous activation using the behaviour selection policy. In real robot experiments, our approach achieves a success rate of 73% with manual activation, and 40% with autonomous behaviour selection. While we show the feasibility of our approach in simulation, the difference in performance between simulated and real world scenarios highlight the difficulty of direct sim-to-real transfer for deep reinforcement learning policies. In both the simulated and real world environments alternative methods were unable to traverse the gap.