mlsanityData to 7 Oct 2026

Deep Bayesian Future Fusion for Self-Supervised, High-Resolution, Off-Road Mapping

arXiv 2403.11876 ↗First posted 18 Mar 2024cs.RO cs.CV cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

High-speed off-road navigation requires long-range, high-resolution maps to enable robots to safely navigate over different surfaces while avoiding dangerous obstacles. However, due to limited computational power and sensing noise, most approaches to off-road mapping focus on producing coarse (20-40cm) maps of the environment. In this paper, we propose Future Fusion, a framework capable of generating dense, high-resolution maps from sparse sensing data (30m forward at 2cm). This is accomplished by - (1) the efficient realization of the well-known Bayes filtering within the standard deep learning models that explicitly accounts for the sparsity pattern in stereo and LiDAR depth data, and (2) leveraging perceptual losses common in generative image completion. The proposed methodology outperforms the conventional baselines. Moreover, the learned features and the completed dense maps lead to improvements in the downstream navigation task.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 34 of its 48 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

34 of 48 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  9. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  10. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  11. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  12. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
22 more
  1. 2017Sparsity Invariant CNNscited by 325
  2. 2020A U-Net Based Discriminator for Generative Adversarial Networkscited by 128
  3. 2020Contextual Residual Aggregation for Ultra High-Resolution Image Inpaintingcited by 124
  4. 2020TartanVO: A Generalizable Learning-based VOcited by 107
  5. 2023CompletionFormer: Depth Completion with Convolutions and Vision Transformerscited by 92
  6. 2019How do neural networks see depth in single images?cited by 82
  7. 2021RigNet: Repetitive Image Guided Network for Depth Completioncited by 80
  8. 2023Point Cloud Forecasting as a Proxy for 4D Occupancy Forecastingcited by 76
  9. 2022Dynamic Spatial Propagation Network for Depth Completioncited by 70
  10. 2022Neural Texture Extraction and Distribution for Controllable Person Image Synthesiscited by 55
  11. 2021GANav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environmentscited by 51
  12. 2020Bidirectional Attention Network for Monocular Depth Estimationcited by 48
  13. 2022TartanDrive: A Large-Scale Dataset for Learning Off-Road Dynamics Modelscited by 48
  14. 2021Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networkscited by 42
  15. 2023TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigationcited by 28
  16. 2021Super Odometry: IMU-centric LiDAR-Visual-Inertial Estimator for Challenging Environmentscited by 26
  17. 2019Visualization of Convolutional Neural Networks for Monocular Depth Estimationcited by 24
  18. 2022How Does It Feel? Self-Supervised Costmap Learning for Off-Road Vehicle Traversabilitycited by 24
  19. 2024RoadRunner -- Learning Traversability Estimation for Autonomous Off-road Drivingcited by 23
  20. 2024TartanDrive 2.0: More Modalities and Better Infrastructure to Further Self-Supervised Learning Research in Off-Road Driving Taskscited by 22
  21. 2023STEP: Stochastic Traversability Evaluation and Planning for Risk-Aware Off-road Navigation; Results from the DARPA Subterranean Challengecited by 11
  22. 2023Learning-on-the-Drive: Self-supervised Adaptation of Visual Offroad Traversability Modelscited by 11

Built on by

Cited by year

Citing papers per year: 2024: 3.

3 papers.

  1. 2024MapEx: Indoor Structure Exploration with Probabilistic Information Gain from Global Map Predictionscited by 9
  2. 2024Map It Anywhere (MIA): Empowering Bird's Eye View Mapping using Large-scale Public Datacited by 5
  3. 2024RoadRunner M&M -- Learning Multi-range Multi-resolution Traversability Maps for Autonomous Off-road Navigationcited by 5