# Synthetic data generation for end-to-end thermal infrared tracking

arXiv: https://arxiv.org/abs/1806.01013 · HTML: https://mlsanity.com/paper/1806.01013/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2018.

## Built on by (most cited first, 26 of 26)

- [LasHeR: A Large-scale High-diversity Benchmark for RGBT Tracking](https://mlsanity.com/paper/2104.13202/) (2021)
- [XoFTR: Cross-modal Feature Matching Transformer](https://mlsanity.com/paper/2404.09692/) (2024)
- [Edge-guided Multi-domain RGB-to-TIR image Translation for Training Vision Tasks with Challenging Labels](https://mlsanity.com/paper/2301.12689/) (2023)
- [Exploring Fusion Strategies for Accurate RGBT Visual Object Tracking](https://mlsanity.com/paper/2201.08673/) (2022)
- [Book2Dial: Generating Teacher-Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbots](https://mlsanity.com/paper/2403.03307/) (2024)
- [Synthetic Test Collections for Retrieval Evaluation](https://mlsanity.com/paper/2405.07767/) (2024)
- [PID: Physics-Informed Diffusion Model for Infrared Image Generation](https://mlsanity.com/paper/2407.09299/) (2024)
- [DiffV2IR: Visible-to-Infrared Diffusion Model via Vision-Language Understanding](https://mlsanity.com/paper/2503.19012/) (2025)
- [Dynamic Fusion Network for RGBT Tracking](https://mlsanity.com/paper/2109.07662/) (2021)
- [Robust pedestrian detection in thermal imagery using synthesized images](https://mlsanity.com/paper/2102.02005/) (2021)
- [Unsupervised Cross-Modal Distillation for Thermal Infrared Tracking](https://mlsanity.com/paper/2108.00187/) (2021)
- [Temporal Aggregation for Adaptive RGBT Tracking](https://mlsanity.com/paper/2201.08949/) (2022)
- [Online Self-Supervised Thermal Water Segmentation for Aerial Vehicles](https://mlsanity.com/paper/2307.09027/) (2023)
- [Adaptive Siamese Tracking with a Compact Latent Network](https://mlsanity.com/paper/2302.00930/) (2023)
- [In Defense and Revival of Bayesian Filtering for Thermal Infrared Object Tracking](https://mlsanity.com/paper/2402.17098/) (2024)
- [Cross-Modal Spherical Aggregation for Weakly Supervised Remote Sensing Shadow Removal](https://mlsanity.com/paper/2406.17469/) (2024)
- [Physically-admissible polarimetric data augmentation for road-scene analysis](https://mlsanity.com/paper/2206.07431/) (2022)
- [Progressive Domain Adaptation for Thermal Infrared Object Tracking](https://mlsanity.com/paper/2407.19430/) (2024)
- [A Comprehensive Survey on Synthetic Infrared Image synthesis](https://mlsanity.com/paper/2408.06868/) (2024)
- [SMMT: Siamese Motion Mamba with Self-attention for Thermal Infrared Target Tracking](https://mlsanity.com/paper/2505.04088/) (2025)
- [PDIWS: Thermal Imaging Dataset for Person Detection in Intrusion Warning Systems](https://mlsanity.com/paper/2302.13293/) (2023)
- [Searching a Lightweight Network Architecture for Thermal Infrared Pedestrian Tracking](https://mlsanity.com/paper/2402.16570/) (2024)
- [Coordinate-Aware Thermal Infrared Tracking Via Natural Language Modeling](https://mlsanity.com/paper/2407.08265/) (2024)
- [RAMCT: Novel Region-adaptive Multi-channel Tracker with Iterative Tikhonov Regularization for Thermal Infrared Tracking](https://mlsanity.com/paper/2504.14278/) (2025)
- [DCFG: Diverse Cross-Channel Fine-Grained Feature Learning and Progressive Fusion Siamese Tracker for Thermal Infrared Target Tracking](https://mlsanity.com/paper/2504.14311/) (2025)
- [RGB-to-IR image translation for infrared vehicle detection in unseen UAV domains](https://mlsanity.com/paper/2609.02556/) (2026)

## Abstract (arXiv)

The usage of both off-the-shelf and end-to-end trained deep networks have significantly improved performance of visual tracking on RGB videos. However, the lack of large labeled datasets hampers the usage of convolutional neural networks for tracking in thermal infrared (TIR) images. Therefore, most state of the art methods on tracking for TIR data are still based on handcrafted features. To address this problem, we propose to use image-to-image translation models. These models allow us to translate the abundantly available labeled RGB data to synthetic TIR data. We explore both the usage of paired and unpaired image translation models for this purpose. These methods provide us with a large labeled dataset of synthetic TIR sequences, on which we can train end-to-end optimal features for tracking. To the best of our knowledge we are the first to train end-to-end features for TIR tracking. We perform extensive experiments on VOT-TIR2017 dataset. We show that a network trained on a large dataset of synthetic TIR data obtains better performance than one trained on the available real TIR data. Combining both data sources leads to further improvement. In addition, when we combine the network with motion features we outperform the state of the art with a relative gain of over 10%, clearly showing the efficiency of using synthetic data to train end-to-end TIR trackers.