# LasHeR: A Large-scale High-diversity Benchmark for RGBT Tracking

arXiv: https://arxiv.org/abs/2104.13202 · HTML: https://mlsanity.com/paper/2104.13202/

## Key facts

- Cited by 96 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 67 references that are in the map.
- First posted on arXiv 27 Apr 2021.

## Builds on (most cited first, 22 of 22)

- [LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking](https://mlsanity.com/paper/1809.07845/) (2018)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild](https://mlsanity.com/paper/1803.10794/) (2018)
- [ATOM: Accurate Tracking by Overlap Maximization](https://mlsanity.com/paper/1811.07628/) (2018)
- [Probabilistic Regression for Visual Tracking](https://mlsanity.com/paper/2003.12565/) (2020)
- [Learning Multi-Domain Convolutional Neural Networks for Visual Tracking](https://mlsanity.com/paper/1510.07945/) (2015)
- [SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines](https://mlsanity.com/paper/1911.06188/) (2019)
- [RGB-T Object Tracking:Benchmark and Baseline](https://mlsanity.com/paper/1805.08982/) (2018)
- [Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration](https://mlsanity.com/paper/1807.03361/) (2018)
- [Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation](https://mlsanity.com/paper/2003.08073/) (2020)
- [Multi-Modal Fusion for End-to-End RGB-T Tracking](https://mlsanity.com/paper/1908.11714/) (2019)
- [Content-Aware Unsupervised Deep Homography Estimation](https://mlsanity.com/paper/1909.05983/) (2019)
- [Deep Homography Estimation for Dynamic Scenes](https://mlsanity.com/paper/2004.02132/) (2020)
- [Challenge-Aware RGBT Tracking](https://mlsanity.com/paper/2007.13143/) (2020)
- [Dual-Stream Pyramid Registration Network](https://mlsanity.com/paper/1909.11966/) (2019)
- [Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking](https://mlsanity.com/paper/2007.02041/) (2020)
- [Dense Feature Aggregation and Pruning for RGBT Tracking](https://mlsanity.com/paper/1907.10451/) (2019)
- [Multi-Adapter RGBT Tracking](https://mlsanity.com/paper/1907.07485/) (2019)
- [RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss](https://mlsanity.com/paper/2011.07189/) (2020)
- [Synthetic data generation for end-to-end thermal infrared tracking](https://mlsanity.com/paper/1806.01013/) (2018)
- [M5L: Multi-Modal Multi-Margin Metric Learning for RGBT Tracking](https://mlsanity.com/paper/2003.07650/) (2020)

## Built on by (most cited first, 30 of 96)

- [Visual Prompt Multi-Modal Tracking](https://mlsanity.com/paper/2303.10826/) (2023)
- [Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline](https://mlsanity.com/paper/2204.04120/) (2022)
- [LRRNet: A Novel Representation Learning Guided Fusion Network for Infrared and Visible Images](https://mlsanity.com/paper/2304.05172/) (2023)
- [Single-Model and Any-Modality for Video Object Tracking](https://mlsanity.com/paper/2311.15851/) (2023)
- [OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning](https://mlsanity.com/paper/2403.09634/) (2024)
- [Prompting for Multi-Modal Tracking](https://mlsanity.com/paper/2207.14571/) (2022)
- [Bi-directional Adapter for Multi-modal Tracking](https://mlsanity.com/paper/2312.10611/) (2023)
- [SDSTrack: Self-Distillation Symmetric Adapter Learning for Multi-Modal Visual Object Tracking](https://mlsanity.com/paper/2403.16002/) (2024)
- [Generative-based Fusion Mechanism for Multi-Modal Tracking](https://mlsanity.com/paper/2309.01728/) (2023)
- [Temporal Adaptive RGBT Tracking with Modality Prompt](https://mlsanity.com/paper/2401.01244/) (2024)
- [Towards Universal Modal Tracking with Online Dense Temporal Token Learning](https://mlsanity.com/paper/2507.20177/) (2025)
- [Visible-Thermal Tiny Object Detection: A Benchmark Dataset and Baselines](https://mlsanity.com/paper/2406.14482/) (2024)
- [Exploring Fusion Strategies for Accurate RGBT Visual Object Tracking](https://mlsanity.com/paper/2201.08673/) (2022)
- [Unified Sequence-to-Sequence Learning for Single- and Multi-Modal Visual Object Tracking](https://mlsanity.com/paper/2304.14394/) (2023)
- [XTrack: Multimodal Training Boosts RGB-X Video Object Trackers](https://mlsanity.com/paper/2405.17773/) (2024)
- [Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarks](https://mlsanity.com/paper/2312.16244/) (2023)
- [RGBT Tracking via All-layer Multimodal Interactions with Progressive Fusion Mamba](https://mlsanity.com/paper/2408.08827/) (2024)
- [What You Have is What You Track: Adaptive and Robust Multimodal Tracking](https://mlsanity.com/paper/2507.05899/) (2025)
- [Cross-modulated Attention Transformer for RGBT Tracking](https://mlsanity.com/paper/2408.02222/) (2024)
- [MambaVT: Spatio-Temporal Contextual Modeling for robust RGB-T Tracking](https://mlsanity.com/paper/2408.07889/) (2024)
- [Visible-Thermal Multiple Object Tracking: Large-scale Video Dataset and Progressive Fusion Approach](https://mlsanity.com/paper/2408.00969/) (2024)
- [RGBD Object Tracking: An In-depth Review](https://mlsanity.com/paper/2203.14134/) (2022)
- [Transformer RGBT Tracking with Spatio-Temporal Multimodal Tokens](https://mlsanity.com/paper/2401.01674/) (2024)
- [Breaking Modality Gap in RGBT Tracking: Coupled Knowledge Distillation](https://mlsanity.com/paper/2410.11586/) (2024)
- [Adaptive Perception for Unified Visual Multi-modal Object Tracking](https://mlsanity.com/paper/2502.06583/) (2025)
- [CADTrack: Learning Contextual Aggregation with Deformable Alignment for Robust RGBT Tracking](https://mlsanity.com/paper/2511.17967/) (2025)
- [CSTrack: Enhancing RGB-X Tracking via Compact Spatiotemporal Features](https://mlsanity.com/paper/2505.19434/) (2025)
- [What Happened 3 Seconds Ago? Inferring the Past with Thermal Imaging](https://mlsanity.com/paper/2304.13651/) (2023)
- [RGB-T Tracking via Multi-Modal Mutual Prompt Learning](https://mlsanity.com/paper/2308.16386/) (2023)
- [Cross-Modal Object Tracking via Modality-Aware Fusion Network and A Large-Scale Dataset](https://mlsanity.com/paper/2312.14446/) (2023)

## Abstract (arXiv)

RGBT tracking receives a surge of interest in the computer vision community, but this research field lacks a large-scale and high-diversity benchmark dataset, which is essential for both the training of deep RGBT trackers and the comprehensive evaluation of RGBT tracking methods. To this end, we present a Large-scale High-diversity benchmark for RGBT tracking (LasHeR) in this work. LasHeR consists of 1224 visible and thermal infrared video pairs with more than 730K frame pairs in total. Each frame pair is spatially aligned and manually annotated with a bounding box, making the dataset well and densely annotated. LasHeR is highly diverse capturing from a broad range of object categories, camera viewpoints, scene complexities and environmental factors across seasons, weathers, day and night. We conduct a comprehensive performance evaluation of 12 RGBT tracking algorithms on the LasHeR dataset and present detailed analysis to clarify the research room in RGBT tracking. In addition, we release the unaligned version of LasHeR to attract the research interest for alignment-free RGBT tracking, which is a more practical task in real-world applications. The datasets and evaluation protocols are available at: https://github.com/BUGPLEASEOUT/LasHeR.