# SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentation for Accurate Freespace Detection

arXiv: https://arxiv.org/abs/2008.11351 · HTML: https://mlsanity.com/paper/2008.11351/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2020.

## Built on by (most cited first, 30 of 66)

- [Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation](https://mlsanity.com/paper/2404.15506/) (2024)
- [Semantics for Robotic Mapping, Perception and Interaction: A Survey](https://mlsanity.com/paper/2101.00443/) (2021)
- [Shape from Polarization for Complex Scenes in the Wild](https://mlsanity.com/paper/2112.11377/) (2021)
- [ORFD: A Dataset and Benchmark for Off-Road Freespace Detection](https://mlsanity.com/paper/2206.09907/) (2022)
- [Audio-Visual Segmentation by Exploring Cross-Modal Mutual Semantics](https://mlsanity.com/paper/2307.16620/) (2023)
- [PVStereo: Pyramid Voting Module for End-to-End Self-Supervised Stereo Matching](https://mlsanity.com/paper/2103.07094/) (2021)
- [RoadFormer: Duplex Transformer for RGB-Normal Semantic Road Scene Parsing](https://mlsanity.com/paper/2309.10356/) (2023)
- [Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation](https://mlsanity.com/paper/2104.12861/) (2021)
- [Dynamic Fusion Module Evolves Drivable Area and Road Anomaly Detection: A Benchmark and Algorithms](https://mlsanity.com/paper/2103.02433/) (2021)
- [SNE-RoadSeg+: Rethinking Depth-Normal Translation and Deep Supervision for Freespace Detection](https://mlsanity.com/paper/2107.14599/) (2021)
- [Playing to Vision Foundation Model's Strengths in Stereo Matching](https://mlsanity.com/paper/2404.06261/) (2024)
- [Computer Vision for Road Imaging and Pothole Detection: A State-of-the-Art Review of Systems and Algorithms](https://mlsanity.com/paper/2204.13590/) (2022)
- [A Simple and Efficient Multi-task Network for 3D Object Detection and Road Understanding](https://mlsanity.com/paper/2103.04056/) (2021)
- [End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Driving](https://mlsanity.com/paper/2104.08862/) (2021)
- [Fast Road Segmentation via Uncertainty-aware Symmetric Network](https://mlsanity.com/paper/2203.04537/) (2022)
- [S3M-Net: Joint Learning of Semantic Segmentation and Stereo Matching for Autonomous Driving](https://mlsanity.com/paper/2401.11414/) (2024)
- [Online,Target-Free LiDAR-Camera Extrinsic Calibration via Cross-Modal Mask Matching](https://mlsanity.com/paper/2404.18083/) (2024)
- [Beyond RGB: Scene-Property Synthesis with Neural Radiance Fields](https://mlsanity.com/paper/2206.04669/) (2022)
- [D2NT: A High-Performing Depth-to-Normal Translator](https://mlsanity.com/paper/2304.12031/) (2023)
- [RoadFormer+: Delivering RGB-X Scene Parsing through Scale-Aware Information Decoupling and Advanced Heterogeneous Feature Fusion](https://mlsanity.com/paper/2407.21631/) (2024)
- [SNE-RoadSegV2: Advancing Heterogeneous Feature Fusion and Fallibility Awareness for Freespace Detection](https://mlsanity.com/paper/2402.18918/) (2024)
- [LIX: Implicitly Infusing Spatial Geometric Prior Knowledge into Visual Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2403.08215/) (2024)
- [SCIPaD: Incorporating Spatial Clues into Unsupervised Pose-Depth Joint Learning](https://mlsanity.com/paper/2407.05283/) (2024)
- [Autonomous Driving in Unstructured Environments: How Far Have We Come?](https://mlsanity.com/paper/2410.07701/) (2024)
- [Pseudo-LiDAR Based Road Detection](https://mlsanity.com/paper/2107.13279/) (2021)
- [Multi-Modal Multi-Task (3MT) Road Segmentation](https://mlsanity.com/paper/2308.11983/) (2023)
- [Scene Prior Filtering for Depth Super-Resolution](https://mlsanity.com/paper/2402.13876/) (2024)
- [ROD: RGB-Only Fast and Efficient Off-road Freespace Detection](https://mlsanity.com/paper/2508.08697/) (2025)
- [SCV-Stereo: Learning Stereo Matching from a Sparse Cost Volume](https://mlsanity.com/paper/2107.08187/) (2021)
- [Monocular BEV Perception of Road Scenes via Front-to-Top View Projection](https://mlsanity.com/paper/2211.08144/) (2022)

## Abstract (arXiv)

Freespace detection is an essential component of visual perception for self-driving cars. The recent efforts made in data-fusion convolutional neural networks (CNNs) have significantly improved semantic driving scene segmentation. Freespace can be hypothesized as a ground plane, on which the points have similar surface normals. Hence, in this paper, we first introduce a novel module, named surface normal estimator (SNE), which can infer surface normal information from dense depth/disparity images with high accuracy and efficiency. Furthermore, we propose a data-fusion CNN architecture, referred to as RoadSeg, which can extract and fuse features from both RGB images and the inferred surface normal information for accurate freespace detection. For research purposes, we publish a large-scale synthetic freespace detection dataset, named Ready-to-Drive (R2D) road dataset, collected under different illumination and weather conditions. The experimental results demonstrate that our proposed SNE module can benefit all the state-of-the-art CNNs for freespace detection, and our SNE-RoadSeg achieves the best overall performance among different datasets.