# PST900: RGB-Thermal Calibration, Dataset and Segmentation Network

arXiv: https://arxiv.org/abs/1909.10980 · HTML: https://mlsanity.com/paper/1909.10980/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2019.

## Built on by (most cited first, 30 of 85)

- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2303.11797/) (2023)
- [Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation](https://mlsanity.com/paper/2308.02097/) (2023)
- [Delivering Arbitrary-Modal Semantic Segmentation](https://mlsanity.com/paper/2303.01480/) (2023)
- [Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation](https://mlsanity.com/paper/2404.04256/) (2024)
- [FEANet: Feature-Enhanced Attention Network for RGB-Thermal Real-time Semantic Segmentation](https://mlsanity.com/paper/2110.08988/) (2021)
- [Edge-aware Guidance Fusion Network for RGB Thermal Scene Parsing](https://mlsanity.com/paper/2112.05144/) (2021)
- [Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks](https://mlsanity.com/paper/2303.15710/) (2023)
- [RGB-T Semantic Segmentation with Location, Activation, and Sharpening](https://mlsanity.com/paper/2210.14530/) (2022)
- [StitchFusion: Weaving Any Visual Modalities to Enhance Multimodal Semantic Segmentation](https://mlsanity.com/paper/2408.01343/) (2024)
- [Learning Modality-agnostic Representation for Semantic Segmentation from Any Modalities](https://mlsanity.com/paper/2407.11351/) (2024)
- [Cross-Spectral Neural Radiance Fields](https://mlsanity.com/paper/2209.00648/) (2022)
- [Complementary Random Masking for RGB-Thermal Semantic Segmentation](https://mlsanity.com/paper/2303.17386/) (2023)
- [Glass Segmentation with RGB-Thermal Image Pairs](https://mlsanity.com/paper/2204.05453/) (2022)
- [Efficient Multimodal Semantic Segmentation via Dual-Prompt Learning](https://mlsanity.com/paper/2312.00360/) (2023)
- [What a MESS: Multi-Domain Evaluation of Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/2306.15521/) (2023)
- [Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentation](https://mlsanity.com/paper/2407.11344/) (2024)
- [Context-Aware Interaction Network for RGB-T Semantic Segmentation](https://mlsanity.com/paper/2401.01624/) (2024)
- [Caltech Aerial RGB-Thermal Dataset in the Wild](https://mlsanity.com/paper/2403.08997/) (2024)
- [Self-Supervised Depth and Ego-Motion Estimation for Monocular Thermal Video Using Multi-Spectral Consistency Loss](https://mlsanity.com/paper/2103.00760/) (2021)
- [Semantic Segmentation for Thermal Images: A Comparative Survey](https://mlsanity.com/paper/2205.13278/) (2022)
- [Thermal3D-GS: Physics-induced 3D Gaussians for Thermal Infrared Novel-view Synthesis](https://mlsanity.com/paper/2409.08042/) (2024)
- [From SLAM to Situational Awareness: Challenges and Survey](https://mlsanity.com/paper/2110.00273/) (2021)
- [Exploring Multi-modal Neural Scene Representations With Applications on Thermal Imaging](https://mlsanity.com/paper/2403.11865/) (2024)
- [RGB-D And Thermal Sensor Fusion: A Systematic Literature Review](https://mlsanity.com/paper/2305.11427/) (2023)
- [U3M: Unbiased Multiscale Modal Fusion Model for Multimodal Semantic Segmentation](https://mlsanity.com/paper/2405.15365/) (2024)
- [Visible-Thermal Multiple Object Tracking: Large-scale Video Dataset and Progressive Fusion Approach](https://mlsanity.com/paper/2408.00969/) (2024)
- [MMSFormer: Multimodal Transformer for Material and Semantic Segmentation](https://mlsanity.com/paper/2309.04001/) (2023)
- [MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selection](https://mlsanity.com/paper/2412.16876/) (2024)
- [Residual Spatial Fusion Network for RGB-Thermal Semantic Segmentation](https://mlsanity.com/paper/2306.10364/) (2023)

## Abstract (arXiv)

In this work we propose long wave infrared (LWIR) imagery as a viable supporting modality for semantic segmentation using learning-based techniques. We first address the problem of RGB-thermal camera calibration by proposing a passive calibration target and procedure that is both portable and easy to use. Second, we present PST900, a dataset of 894 synchronized and calibrated RGB and Thermal image pairs with per pixel human annotations across four distinct classes from the DARPA Subterranean Challenge. Lastly, we propose a CNN architecture for fast semantic segmentation that combines both RGB and Thermal imagery in a way that leverages RGB imagery independently. We compare our method against the state-of-the-art and show that our method outperforms them in our dataset.