# RGB-Thermal Infrared Fusion for Robust Depth Estimation in Complex Environments

arXiv: https://arxiv.org/abs/2503.04821 · HTML: https://mlsanity.com/paper/2503.04821/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 5 Mar 2025.

## Builds on (most cited first, 15 of 15)

- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation](https://mlsanity.com/paper/2312.02145/) (2023)
- [Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation](https://mlsanity.com/paper/2203.11483/) (2022)
- [Revealing the Dark Secrets of Masked Image Modeling](https://mlsanity.com/paper/2205.13543/) (2022)
- [ViViD++: Vision for Visibility Dataset](https://mlsanity.com/paper/2204.06183/) (2022)
- [Adaptive Fusion of Single-View and Multi-View Depth for Autonomous Driving](https://mlsanity.com/paper/2403.07535/) (2024)
- [Self-Supervised Depth and Ego-Motion Estimation for Monocular Thermal Video Using Multi-Spectral Consistency Loss](https://mlsanity.com/paper/2103.00760/) (2021)
- [Maximizing Self-supervision from Thermal Image for Effective Self-supervised Learning of Depth and Ego-motion](https://mlsanity.com/paper/2201.04387/) (2022)
- [Fieldscale: Locality-Aware Field-based Adaptive Rescaling for Thermal Infrared Image](https://mlsanity.com/paper/2405.15395/) (2024)
- [Dusk Till Dawn: Self-supervised Nighttime Stereo Depth Estimation using Visual Foundation Models](https://mlsanity.com/paper/2405.11158/) (2024)
- [Unveiling the Depths: A Multi-Modal Fusion Framework for Challenging Scenarios](https://mlsanity.com/paper/2402.11826/) (2024)
- [Adaptive Stereo Depth Estimation with Multi-Spectral Images Across All Lighting Conditions](https://mlsanity.com/paper/2411.03638/) (2024)

## Built on by (most cited first, 1 of 1)

- [Propagating Sparse Depth via Depth Foundation Model for Out-of-Distribution Depth Completion](https://mlsanity.com/paper/2508.04984/) (2025)

## Abstract (arXiv)

Depth estimation in complex real-world scenarios is a challenging task, especially when relying solely on a single modality such as visible light or thermal infrared (THR) imagery. This paper proposes a novel multimodal depth estimation model, RTFusion, which enhances depth estimation accuracy and robustness by integrating the complementary strengths of RGB and THR data. The RGB modality provides rich texture and color information, while the THR modality captures thermal patterns, ensuring stability under adverse lighting conditions such as extreme illumination. The model incorporates a unique fusion mechanism, EGFusion, consisting of the Mutual Complementary Attention (MCA) module for cross-modal feature alignment and the Edge Saliency Enhancement Module (ESEM) to improve edge detail preservation. Comprehensive experiments on the MS2 and ViViD++ datasets demonstrate that the proposed model consistently produces high-quality depth maps across various challenging environments, including nighttime, rainy, and high-glare conditions. The experimental results highlight the potential of the proposed method in applications requiring reliable depth estimation, such as autonomous driving, robotics, and augmented reality.