# Graph-based Thermal-Inertial SLAM with Probabilistic Neural Networks

arXiv: https://arxiv.org/abs/2104.07196 · HTML: https://mlsanity.com/paper/2104.07196/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 95 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age](https://mlsanity.com/paper/1606.05830/) (2016)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks](https://mlsanity.com/paper/1709.08429/) (2017)
- [Modelling Uncertainty in Deep Learning for Camera Relocalization](https://mlsanity.com/paper/1509.05909/) (2015)
- [DeepFactors: Real-Time Probabilistic Dense Monocular SLAM](https://mlsanity.com/paper/2001.05049/) (2020)
- [Deep Learning Features at Scale for Visual Place Recognition](https://mlsanity.com/paper/1701.05105/) (2017)
- [UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning](https://mlsanity.com/paper/1709.06841/) (2017)
- [AtLoc: Attention Guided Camera Localization](https://mlsanity.com/paper/1909.03557/) (2019)
- [Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network](https://mlsanity.com/paper/1707.09733/) (2017)
- [LO-Net: Deep Real-time Lidar Odometry](https://mlsanity.com/paper/1904.08242/) (2019)
- [VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization](https://mlsanity.com/paper/1702.06521/) (2017)
- [VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem](https://mlsanity.com/paper/1701.08376/) (2017)
- [SegMatch: Segment based loop-closure for 3D point clouds](https://mlsanity.com/paper/1609.07720/) (2016)
- [Toward Geometric Deep SLAM](https://mlsanity.com/paper/1707.07410/) (2017)
- [Selective Sensor Fusion for Neural Visual-Inertial Odometry](https://mlsanity.com/paper/1903.01534/) (2019)
- [Distilling Knowledge From a Deep Pose Regressor Network](https://mlsanity.com/paper/1908.00858/) (2019)
- [Learning to Localize Using a LiDAR Intensity Map](https://mlsanity.com/paper/2012.10902/) (2020)
- [Sparse Depth Enhanced Direct Thermal-infrared SLAM Beyond the Visible Spectrum](https://mlsanity.com/paper/1902.10892/) (2019)
- [GANVO: Unsupervised Deep Monocular Visual Odometry and Depth Estimation with Generative Adversarial Networks](https://mlsanity.com/paper/1809.05786/) (2018)
- [DeepTIO: A Deep Thermal-Inertial Odometry with Visual Hallucination](https://mlsanity.com/paper/1909.07231/) (2019)
- [VMLoc: Variational Fusion For Learning-Based Multimodal Camera Localization](https://mlsanity.com/paper/2003.07289/) (2020)

## Built on by (most cited first, 12 of 12)

- [Edge-guided Multi-domain RGB-to-TIR image Translation for Training Vision Tasks with Challenging Labels](https://mlsanity.com/paper/2301.12689/) (2023)
- [Caltech Aerial RGB-Thermal Dataset in the Wild](https://mlsanity.com/paper/2403.08997/) (2024)
- [From SLAM to Situational Awareness: Challenges and Survey](https://mlsanity.com/paper/2110.00273/) (2021)
- [Fieldscale: Locality-Aware Field-based Adaptive Rescaling for Thermal Infrared Image](https://mlsanity.com/paper/2405.15395/) (2024)
- [ThermoHands: A Benchmark for 3D Hand Pose Estimation from Egocentric Thermal Images](https://mlsanity.com/paper/2403.09871/) (2024)
- [Towards Long Term SLAM on Thermal Imagery](https://mlsanity.com/paper/2403.19885/) (2024)
- [MMOne: Representing Multiple Modalities in One Scene](https://mlsanity.com/paper/2507.11129/) (2025)
- [MrGS: Multi-modal Radiance Fields with 3D Gaussian Splatting for RGB-Thermal Novel View Synthesis](https://mlsanity.com/paper/2511.22997/) (2025)
- [ThermalLoc: A Vision Transformer-Based Approach for Robust Thermal Camera Relocalization in Large-Scale Environments](https://mlsanity.com/paper/2506.18268/) (2025)
- [Thermal odometry and dense mapping using learned odometry and Gaussian splatting](https://mlsanity.com/paper/2602.07493/) (2026)
- [Thegra: Graph-based SLAM for Thermal Imagery](https://mlsanity.com/paper/2602.08531/) (2026)
- [LST-SLAM: A Stereo Thermal SLAM System for Kilometer-Scale Dynamic Environments](https://mlsanity.com/paper/2602.20925/) (2026)

## Abstract (arXiv)

Simultaneous Localization and Mapping (SLAM) system typically employ vision-based sensors to observe the surrounding environment. However, the performance of such systems highly depends on the ambient illumination conditions. In scenarios with adverse visibility or in the presence of airborne particulates (e.g. smoke, dust, etc.), alternative modalities such as those based on thermal imaging and inertial sensors are more promising. In this paper, we propose the first complete thermal-inertial SLAM system which combines neural abstraction in the SLAM front end with robust pose graph optimization in the SLAM back end. We model the sensor abstraction in the front end by employing probabilistic deep learning parameterized by Mixture Density Networks (MDN). Our key strategies to successfully model this encoding from thermal imagery are the usage of normalized 14-bit radiometric data, the incorporation of hallucinated visual (RGB) features, and the inclusion of feature selection to estimate the MDN parameters. To enable a full SLAM system, we also design an efficient global image descriptor which is able to detect loop closures from thermal embedding vectors. We performed extensive experiments and analysis using three datasets, namely self-collected ground robot and handheld data taken in indoor environment, and one public dataset (SubT-tunnel) collected in underground tunnel. Finally, we demonstrate that an accurate thermal-inertial SLAM system can be realized in conditions of both benign and adverse visibility.