mlsanityData to 7 Oct 2026

UFO: Uncertainty-aware LiDAR-image Fusion for Off-road Semantic Terrain Map Estimation

arXiv 2403.02642 ↗First posted 5 Mar 2024cs.RO cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Autonomous off-road navigation requires an accurate semantic understanding of the environment, often converted into a bird's-eye view (BEV) representation for various downstream tasks. While learning-based methods have shown success in generating local semantic terrain maps directly from sensor data, their efficacy in off-road environments is hindered by challenges in accurately representing uncertain terrain features. This paper presents a learning-based fusion method for generating dense terrain classification maps in BEV. By performing LiDAR-image fusion at multiple scales, our approach enhances the accuracy of semantic maps generated from an RGB image and a single-sweep LiDAR scan. Utilizing uncertainty-aware pseudo-labels further enhances the network's ability to learn reliably in off-road environments without requiring precise 3D annotations. By conducting thorough experiments using off-road driving datasets, we demonstrate that our method can improve accuracy in off-road terrains, validating its efficacy in facilitating reliable and safe autonomous navigation in challenging off-road settings.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 37 of its 48 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

37 of 48 references.

References · 37

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  3. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  4. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  5. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
  6. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  7. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  8. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  9. 2020LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mappingcited by 185
  10. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
  11. 20222DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Cloudscited by 157
  12. 2020CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detectioncited by 147
25 more
  1. 2022Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?cited by 128
  2. 2020RELLIS-3D Dataset: Data, Benchmarks and Analysiscited by 118
  3. 2021Translating Images into Mapscited by 115
  4. 2020S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Cloudscited by 102
  5. 2023SCPNet: Semantic Scene Completion on Point Cloudcited by 80
  6. 2021EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentationcited by 59
  7. 2021GANav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environmentscited by 51
  8. 2021Sparse-to-dense Feature Matching: Intra and Inter domain Cross-modal Learning in Domain Adaptation for 3D Semantic Segmentationcited by 46
  9. 20214D-Net for Learned Multi-Modal Alignmentcited by 44
  10. 2022MotionSC: Data Set and Network for Real-Time Semantic Mapping in Dynamic Environmentscited by 40
  11. 2019Progressive LiDAR Adaptation for Road Detectioncited by 37
  12. 2019Bayesian Spatial Kernel Smoothing for Scalable Dense Semantic Mappingcited by 34
  13. 2022Elevation Mapping for Locomotion and Navigation using GPUcited by 32
  14. 2022WayFAST: Navigation with Predictive Traversability in the Fieldcited by 30
  15. 2023TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigationcited by 28
  16. 2023Learning Off-Road Terrain Traversability with Self-Supervisions Onlycited by 24
  17. 2021MASS: Multi-Attentional Semantic Segmentation of LiDAR Data for Dense Top-View Understandingcited by 22
  18. 2021Learning with Noisy Labels for Robust Point Cloud Segmentationcited by 22
  19. 2022ScaTE: A Scalable Framework for Self-Supervised Traversability Estimation in Unstructured Environmentscited by 21
  20. 2022Risk-Aware Off-Road Navigation via a Learned Speed Distribution Mapcited by 17
  21. 2021PillarSegNet: Pillar-based Semantic Grid Map Estimation using Sparse LiDAR Datacited by 16
  22. 2022Probabilistic Traversability Model for Risk-Aware Motion Planning in Off-Road Environmentscited by 13
  23. 2022Convolutional Bayesian Kernel Inference for 3D Semantic Mappingcited by 8
  24. 2020Planning Paths Through Unknown Space by Imagining What Lies Thereincited by 7
  25. 2023Multi-Source Soft Pseudo-Label Learning with Domain Similarity-based Weighting for Semantic Segmentationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 3.

5 papers.

  1. 2025Advancing Off-Road Autonomous Driving: The Large-Scale ORAD-3D Dataset and Comprehensive Benchmarkscited by 8
  2. 2024Autonomous Driving in Unstructured Environments: How Far Have We Come?cited by 7
  3. 2026OT-Drive: Out-of-Distribution Off-Road Traversable Area Segmentation via Optimal Transportcited by 0
  4. 2026FusionSense: Tri-Stage Near-Sensor Learning for Runtime-Adaptive Multimodal Edge Intelligencecited by 0
  5. 2026PRISM: Multimodal Terrain Mapping for Rover Navigation in Unstructured Environmentscited by 0