# RGB-D-based Stair Detection using Deep Learning for Autonomous Stair Climbing

arXiv: https://arxiv.org/abs/2212.01098 · HTML: https://mlsanity.com/paper/2212.01098/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 32 references that are in the map.
- First posted on arXiv 2 Dec 2022.

## Builds on (most cited first, 9 of 9)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [A Review on Deep Learning Techniques Applied to Semantic Segmentation](https://mlsanity.com/paper/1704.06857/) (2017)
- [Multimodal Deep Learning for Robust RGB-D Object Recognition](https://mlsanity.com/paper/1507.06821/) (2015)
- [Deep Leaning-Based Ultra-Fast Stair Detection](https://mlsanity.com/paper/2201.05275/) (2022)
- [Unsupervised Domain Adaptation Learning Algorithm for RGB-D Staircase Recognition](https://mlsanity.com/paper/1903.01212/) (2019)
- [Expandable YOLO: 3D Object Detection from RGB-D Images](https://mlsanity.com/paper/2006.14837/) (2020)
- [Review on Indoor RGB-D Semantic Segmentation with Deep Convolutional Neural Networks](https://mlsanity.com/paper/2105.11925/) (2021)

## Abstract (arXiv)

Stairs are common building structures in urban environments, and stair detection is an important part of environment perception for autonomous mobile robots. Most existing algorithms have difficulty combining the visual information from binocular sensors effectively and ensuring reliable detection at night and in the case of extremely fuzzy visual clues. To solve these problems, we propose a neural network architecture with RGB and depth map inputs. Specifically, we design a selective module, which can make the network learn the complementary relationship between the RGB map and the depth map and effectively combine the information from the RGB map and the depth map in different scenes. In addition, we design a line clustering algorithm for the postprocessing of detection results, which can make full use of the detection results to obtain the geometric stair parameters. Experiments on our dataset show that our method can achieve better accuracy and recall compared with existing state-of-the-art deep learning methods, which are 5.64% and 7.97%, respectively, and our method also has extremely fast detection speed. A lightweight version can achieve 300 + frames per second with the same resolution, which can meet the needs of most real-time detection scenes.