# DeepTIO: A Deep Thermal-Inertial Odometry with Visual Hallucination

arXiv: https://arxiv.org/abs/1909.07231 · HTML: https://mlsanity.com/paper/1909.07231/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2019.

## Built on by (most cited first, 15 of 15)

- [Edge-guided Multi-domain RGB-to-TIR image Translation for Training Vision Tasks with Challenging Labels](https://mlsanity.com/paper/2301.12689/) (2023)
- [Deep Learning for Inertial Positioning: A Survey](https://mlsanity.com/paper/2303.03757/) (2023)
- [Self-Supervised Depth and Ego-Motion Estimation for Monocular Thermal Video Using Multi-Spectral Consistency Loss](https://mlsanity.com/paper/2103.00760/) (2021)
- [Graph-based Thermal-Inertial SLAM with Probabilistic Neural Networks](https://mlsanity.com/paper/2104.07196/) (2021)
- [Visible-Infrared Person Re-Identification Using Privileged Intermediate Information](https://mlsanity.com/paper/2209.09348/) (2022)
- [Robust 3D Object Detection from LiDAR-Radar Point Clouds via Cross-Modal Feature Augmentation](https://mlsanity.com/paper/2309.17336/) (2023)
- [Adaptive Generation of Privileged Intermediate Information for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2307.03240/) (2023)
- [NeuV-SLAM: Fast Neural Multiresolution Voxel Optimization for RGBD Dense SLAM](https://mlsanity.com/paper/2402.02020/) (2024)
- [ThermoHands: A Benchmark for 3D Hand Pose Estimation from Egocentric Thermal Images](https://mlsanity.com/paper/2403.09871/) (2024)
- [Thermal Chameleon: Task-Adaptive Tone-mapping for Radiometric Thermal-Infrared images](https://mlsanity.com/paper/2410.18340/) (2024)
- [Uni-SLAM: Uncertainty-Aware Neural Implicit SLAM for Real-Time Dense Indoor Scene Reconstruction](https://mlsanity.com/paper/2412.00242/) (2024)
- [Cross-Modal Contrastive Learning of Representations for Navigation using Lightweight, Low-Cost Millimeter Wave Radar for Adverse Environmental Conditions](https://mlsanity.com/paper/2101.03525/) (2021)
- [ThermalLoc: A Vision Transformer-Based Approach for Robust Thermal Camera Relocalization in Large-Scale Environments](https://mlsanity.com/paper/2506.18268/) (2025)
- [Thegra: Graph-based SLAM for Thermal Imagery](https://mlsanity.com/paper/2602.08531/) (2026)
- [TIDY: Thermal Infrared Image Denoising via Wavelet Domain Entropy and Directional Stripe Index](https://mlsanity.com/paper/2606.19813/) (2026)

## Abstract (arXiv)

Visual odometry shows excellent performance in a wide range of environments. However, in visually-denied scenarios (e.g. heavy smoke or darkness), pose estimates degrade or even fail. Thermal cameras are commonly used for perception and inspection when the environment has low visibility. However, their use in odometry estimation is hampered by the lack of robust visual features. In part, this is as a result of the sensor measuring the ambient temperature profile rather than scene appearance and geometry. To overcome this issue, we propose a Deep Neural Network model for thermal-inertial odometry (DeepTIO) by incorporating a visual hallucination network to provide the thermal network with complementary information. The hallucination network is taught to predict fake visual features from thermal images by using Huber loss. We also employ selective fusion to attentively fuse the features from three different modalities, i.e thermal, hallucination, and inertial features. Extensive experiments are performed in hand-held and mobile robot data in benign and smoke-filled environments, showing the efficacy of the proposed model.