# Real-time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-driving Images

arXiv: https://arxiv.org/abs/2002.10570 · HTML: https://mlsanity.com/paper/2002.10570/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2020.

## Built on by (most cited first, 30 of 36)

- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [Capturing Omni-Range Context for Omnidirectional Segmentation](https://mlsanity.com/paper/2103.05687/) (2021)
- [TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Network](https://mlsanity.com/paper/2108.03990/) (2021)
- [WayFAST: Navigation with Predictive Traversability in the Field](https://mlsanity.com/paper/2203.12071/) (2022)
- [Exploring Event-driven Dynamic Context for Accident Scene Segmentation](https://mlsanity.com/paper/2112.05006/) (2021)
- [Fast Road Segmentation via Uncertainty-aware Symmetric Network](https://mlsanity.com/paper/2203.04537/) (2022)
- [RGB-D And Thermal Sensor Fusion: A Systematic Literature Review](https://mlsanity.com/paper/2305.11427/) (2023)
- [SRFNet: Monocular Depth Estimation with Fine-grained Structure via Spatial Reliability-oriented Fusion of Frames and Events](https://mlsanity.com/paper/2309.12842/) (2023)
- [Visual Prompt Selection for In-Context Learning Segmentation](https://mlsanity.com/paper/2407.10233/) (2024)
- [A Survey of Visual Sensory Anomaly Detection](https://mlsanity.com/paper/2202.07006/) (2022)
- [Indoor Obstacle Discovery on Reflective Ground via Monocular Camera](https://mlsanity.com/paper/2401.01445/) (2024)
- [SNE-RoadSegV2: Advancing Heterogeneous Feature Fusion and Fallibility Awareness for Freespace Detection](https://mlsanity.com/paper/2402.18918/) (2024)
- [Iterative Loop Method Combining Active and Semi-Supervised Learning for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2301.13361/) (2023)
- [ShareCMP: Polarization-Aware RGB-P Semantic Segmentation](https://mlsanity.com/paper/2312.03430/) (2023)
- [Aerial-PASS: Panoramic Annular Scene Segmentation in Drone Videos](https://mlsanity.com/paper/2105.07209/) (2021)
- [Multimodal Detection of Unknown Objects on Roads for Autonomous Driving](https://mlsanity.com/paper/2205.01414/) (2022)
- [Camera-Radar Perception for Autonomous Vehicles and ADAS: Concepts, Datasets and Metrics](https://mlsanity.com/paper/2303.04302/) (2023)
- [Spatially-guided Temporal Aggregation for Robust Event-RGB Optical Flow Estimation](https://mlsanity.com/paper/2501.00838/) (2025)
- [ISETAuto: Detecting vehicles with depth and radiance information](https://mlsanity.com/paper/2101.01843/) (2021)
- [Perception Framework through Real-Time Semantic Segmentation and Scene Recognition on a Wearable System for the Visually Impaired](https://mlsanity.com/paper/2103.04136/) (2021)
- [Stereo Object Matching Network](https://mlsanity.com/paper/2103.12498/) (2021)
- [Panoramic Depth Estimation via Supervised and Unsupervised Learning in Indoor Scenes](https://mlsanity.com/paper/2108.08076/) (2021)
- [Perspective Aware Road Obstacle Detection](https://mlsanity.com/paper/2210.01779/) (2022)
- [Training Semantic Segmentation on Heterogeneous Datasets](https://mlsanity.com/paper/2301.07634/) (2023)
- [A Threefold Review on Deep Semantic Segmentation: Efficiency-oriented, Temporal and Depth-aware design](https://mlsanity.com/paper/2303.04315/) (2023)
- [DeepIPCv2: LiDAR-powered Robust Environmental Perception and Navigational Control for Autonomous Vehicle](https://mlsanity.com/paper/2307.06647/) (2023)
- [LAECIPS: Large Vision Model Assisted Adaptive Edge-Cloud Collaboration for IoT-based Embodied Intelligence System](https://mlsanity.com/paper/2404.10498/) (2024)
- [Edge Case Detection in Automated Driving: Methods, Challenges, and Future Directions](https://mlsanity.com/paper/2410.08491/) (2024)
- [HazardNet: Road Debris Detection by Augmentation of Synthetic Models](https://mlsanity.com/paper/2303.07547/) (2023)
- [General-Purpose Multi-Modal OOD Detection Framework](https://mlsanity.com/paper/2307.13069/) (2023)

## Abstract (arXiv)

Semantic segmentation has made striking progress due to the success of deep convolutional neural networks. Considering the demands of autonomous driving, real-time semantic segmentation has become a research hotspot these years. However, few real-time RGB-D fusion semantic segmentation studies are carried out despite readily accessible depth information nowadays. In this paper, we propose a real-time fusion semantic segmentation network termed RFNet that effectively exploits complementary cross-modal information. Building on an efficient network architecture, RFNet is capable of running swiftly, which satisfies autonomous vehicles applications. Multi-dataset training is leveraged to incorporate unexpected small obstacle detection, enriching the recognizable classes required to face unforeseen hazards in the real world. A comprehensive set of experiments demonstrates the effectiveness of our framework. On Cityscapes, Our method outperforms previous state-of-the-art semantic segmenters, with excellent accuracy and 22Hz inference speed at the full 2048x1024 resolution, outperforming most existing RGB-D networks.