# IRisPath: Enhancing Costmap for Off-Road Navigation with Robust IR-RGB Fusion for Improved Day and Night Traversability

arXiv: https://arxiv.org/abs/2412.03173 · HTML: https://mlsanity.com/paper/2412.03173/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 4 Dec 2024.

## Builds on (most cited first, 20 of 20)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection](https://mlsanity.com/paper/2203.16220/) (2022)
- [LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping](https://mlsanity.com/paper/2007.00258/) (2020)
- [FAST-LIO2: Fast Direct LiDAR-inertial Odometry](https://mlsanity.com/paper/2107.06829/) (2021)
- [RELLIS-3D Dataset: Data, Benchmarks and Analysis](https://mlsanity.com/paper/2011.12954/) (2020)
- [GANav: Efficient Terrain Segmentation for Robot Navigation in Unstructured Outdoor Environments](https://mlsanity.com/paper/2103.04233/) (2021)
- [The GOOSE Dataset for Perception in Unstructured Environments](https://mlsanity.com/paper/2310.16788/) (2023)
- [How Does It Feel? Self-Supervised Costmap Learning for Off-Road Vehicle Traversability](https://mlsanity.com/paper/2209.10788/) (2022)
- [GrASPE: Graph based Multimodal Fusion for Robot Navigation in Unstructured Outdoor Environments](https://mlsanity.com/paper/2209.05722/) (2022)
- [TartanDrive 2.0: More Modalities and Better Infrastructure to Further Self-Supervised Learning Research in Off-Road Driving Tasks](https://mlsanity.com/paper/2402.01913/) (2024)
- [OFFSEG: A Semantic Segmentation Framework For Off-Road Driving](https://mlsanity.com/paper/2103.12417/) (2021)
- [LiDAR-Camera Calibration using 3D-3D Point correspondences](https://mlsanity.com/paper/1705.09785/) (2017)
- [LiDAR and Camera Calibration using Motion Estimated by Sensor Fusion Odometry](https://mlsanity.com/paper/1804.05178/) (2018)
- [Calib-Anything: Zero-training LiDAR-Camera Extrinsic Calibration Method Using Segment Anything](https://mlsanity.com/paper/2306.02656/) (2023)
- [Targetless Extrinsic Calibration of Stereo Cameras, Thermal Cameras, and Laser Sensors in the Wild](https://mlsanity.com/paper/2109.13414/) (2021)
- [FoMo: A Proposal for a Multi-Season Dataset for Robot Navigation in Foret Montmorency](https://mlsanity.com/paper/2404.13166/) (2024)
- [Single-Shot is Enough: Panoramic Infrastructure Based Calibration of Multiple Cameras and 3D LiDARs](https://mlsanity.com/paper/2103.12941/) (2021)
- [MFCalib: Single-shot and Automatic Extrinsic Calibration for LiDAR and Camera in Targetless Environments Based on Multi-Feature Edge](https://mlsanity.com/paper/2409.00992/) (2024)

## Abstract (arXiv)

Autonomous off-road navigation is required for applications in agriculture, construction, search and rescue and defence. Traditional on-road autonomous methods struggle with dynamic terrains, leading to poor vehicle control in off-road conditions. Recent deep-learning models have used perception sensors along with kinesthetic feedback for navigation on such terrains. However, this approach has out-of-domain uncertainty. Factors like change in time of day and weather impacts the performance of the model. We propose a multi modal fusion network "IRisPath" capable of using Thermal and RGB images to provide robustness against dynamic weather and light conditions. To aid further works in this domain, we also open-source a day-night dataset with Thermal and RGB images along with pseudo-labels for traversability. In order to co-register for fusion model we also develop a novel method for targetless extrinsic calibration of Thermal, LiDAR and RGB cameras with translation accuracy of +/-1.7cm and rotation accuracy of +/-0.827degrees.