# How Does It Feel? Self-Supervised Costmap Learning for Off-Road Vehicle Traversability

arXiv: https://arxiv.org/abs/2209.10788 · HTML: https://mlsanity.com/paper/2209.10788/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 55 references that are in the map.
- First posted on arXiv 22 Sep 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Rapid Exploration for Open-World Navigation with Latent Goal Models](https://mlsanity.com/paper/2104.05859/) (2021)
- [TartanVO: A Generalizable Learning-based VO](https://mlsanity.com/paper/2011.00359/) (2020)
- [BADGR: An Autonomous Self-Supervised Learning-Based Navigation System](https://mlsanity.com/paper/2002.05700/) (2020)
- [Coupling Vision and Proprioception for Navigation of Legged Robots](https://mlsanity.com/paper/2112.02094/) (2021)
- [TartanDrive: A Large-Scale Dataset for Learning Off-Road Dynamics Models](https://mlsanity.com/paper/2205.01791/) (2022)
- [Super Odometry: IMU-centric LiDAR-Visual-Inertial Estimator for Challenging Environments](https://mlsanity.com/paper/2104.14938/) (2021)
- [GrASPE: Graph based Multimodal Fusion for Robot Navigation in Unstructured Outdoor Environments](https://mlsanity.com/paper/2209.05722/) (2022)
- [Risk-Aware Off-Road Navigation via a Learned Speed Distribution Map](https://mlsanity.com/paper/2203.13429/) (2022)
- [Learning Risk-aware Costmaps for Traversability in Challenging Environments](https://mlsanity.com/paper/2107.11722/) (2021)
- [RCA: Ride Comfort-Aware Visual Navigation via Self-Supervised Learning](https://mlsanity.com/paper/2207.14460/) (2022)

## Built on by (most cited first, 27 of 27)

- [Learning Off-Road Terrain Traversability with Self-Supervisions Only](https://mlsanity.com/paper/2305.18896/) (2023)
- [RoadRunner -- Learning Traversability Estimation for Autonomous Off-road Driving](https://mlsanity.com/paper/2402.19341/) (2024)
- [Wild Visual Navigation: Fast Traversability Learning via Pre-Trained Models and Online Self-Supervision](https://mlsanity.com/paper/2404.07110/) (2024)
- [Reasoning about the Unseen for Efficient Outdoor Object Navigation](https://mlsanity.com/paper/2309.10103/) (2023)
- [Reinforcement Learning for Collision-free Flight Exploiting Deep Collision Encoding](https://mlsanity.com/paper/2402.03947/) (2024)
- [Learning to See Physical Properties with Active Sensing Motor Policies](https://mlsanity.com/paper/2311.01405/) (2023)
- [Identifying Terrain Physical Parameters from Vision -- Towards Physical-Parameter-Aware Locomotion and Navigation](https://mlsanity.com/paper/2408.16567/) (2024)
- [Reinforcement Learning for Wheeled Mobility on Vertically Challenging Terrain](https://mlsanity.com/paper/2409.02383/) (2024)
- [VAMOS: A Hierarchical Vision-Language-Action Model for Capability-Modulated and Steerable Navigation](https://mlsanity.com/paper/2510.20818/) (2025)
- [Safe Navigation in Unstructured Environments by Minimizing Uncertainty in Control and Perception](https://mlsanity.com/paper/2306.14601/) (2023)
- [Neural Potential Field for Obstacle-Aware Local Motion Planning](https://mlsanity.com/paper/2310.16362/) (2023)
- [Deep Bayesian Future Fusion for Self-Supervised, High-Resolution, Off-Road Mapping](https://mlsanity.com/paper/2403.11876/) (2024)
- [VertiSelector: Automatic Curriculum Learning for Wheeled Mobility on Vertically Challenging Terrain](https://mlsanity.com/paper/2409.17469/) (2024)
- [TADPO: Reinforcement Learning Goes Off-road](https://mlsanity.com/paper/2603.05995/) (2026)
- [FusionForce: End-to-end Differentiable Neural-Symbolic Layer for Trajectory Prediction](https://mlsanity.com/paper/2502.10156/) (2025)
- [T-CBF: Traversability-based Control Barrier Function to Navigate Vertically Challenging Terrain](https://mlsanity.com/paper/2503.06083/) (2025)
- [Motion planning for off-road autonomous driving based on human-like cognition and weight adaptation](https://mlsanity.com/paper/2404.17820/) (2024)
- [Situationally-Aware Dynamics Learning](https://mlsanity.com/paper/2505.19574/) (2025)
- [Synthetic vs. Real Training Data for Visual Navigation](https://mlsanity.com/paper/2509.11791/) (2025)
- [Self-Supervised Online Robot-Agnostic Traversability Estimation for Open-World Environments](https://mlsanity.com/paper/2605.28442/) (2026)
- [METAVerse: Meta-Learning Traversability Cost Map for Off-Road Navigation](https://mlsanity.com/paper/2307.13991/) (2023)
- [IRisPath: Enhancing Costmap for Off-Road Navigation with Robust IR-RGB Fusion for Improved Day and Night Traversability](https://mlsanity.com/paper/2412.03173/) (2024)
- [Mars Traversability Prediction: A Multi-modal Self-supervised Approach for Costmap Generation](https://mlsanity.com/paper/2509.11082/) (2025)
- [ProTerrain: Probabilistic Physics-Informed Rough Terrain World Modeling](https://mlsanity.com/paper/2510.19364/) (2025)
- [Multimodal embodiment-aware navigation transformer](https://mlsanity.com/paper/2604.19267/) (2026)
- [Feeling Terrain Before Crossing: World Models for Off-Road Navigation](https://mlsanity.com/paper/2609.19863/) (2026)
- [FLINT: Fast Lightweight Inference for Traversability](https://mlsanity.com/paper/2609.26857/) (2026)

## Abstract (arXiv)

Estimating terrain traversability in off-road environments requires reasoning about complex interaction dynamics between the robot and these terrains. However, it is challenging to create informative labels to learn a model in a supervised manner for these interactions. We propose a method that learns to predict traversability costmaps by combining exteroceptive environmental information with proprioceptive terrain interaction feedback in a self-supervised manner. Additionally, we propose a novel way of incorporating robot velocity in the costmap prediction pipeline. We validate our method in multiple short and large-scale navigation tasks on challenging off-road terrains using two different large, all-terrain robots. Our short-scale navigation results show that using our learned costmaps leads to overall smoother navigation, and provides the robot with a more fine-grained understanding of the robot-terrain interactions. Our large-scale navigation trials show that we can reduce the number of interventions by up to 57% compared to an occupancy-based navigation baseline in challenging off-road courses ranging from 400 m to 3150 m. Appendix and full experiment videos can be found in our website: https://mateoguaman.github.io/hdif.